Systems, devices, and methods for generating messages

ABSTRACT

Systems, devices, and methods for generating and sending messages are described. messages are autonomously generated by at least one processor by identifying environmental context of a device, system, or user, such that minimal input is required from the user. Environmental context can include at least time, date, calendar entries, user activity, location, physical environment, message history, and relationship between users. Messages can also be generated by generating a first portion of a message, and having the user select a second portion to complete the message. Recipients for a message can be autonomously determined based on environmental context and content of a message to be sent.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/692,484, filed Jun. 29, 2018, titled “Systems, Devices, and Methodsfor Generating Conversation Starter Messages”, the content of which isincorporated herein in its entirety by reference.

BACKGROUND Technical Field

The present systems, devices, and methods generally relate tocommunication content, and particularly, to the generation of messages.

Description of the Related Art

Portable Electronic Devices

Electronic devices are commonplace throughout most of the world today.Advancements in integrated circuit technology have enabled thedevelopment of electronic devices that are sufficiently small andlightweight to be carried by the user. Such “portable” electronicdevices may include on-board power supplies (such as batteries or otherpower storage systems) and may be designed to operate without anywire-connections to other, non-portable electronic systems; however, asmall and lightweight electronic device may still be considered portableeven if it includes a wire-connection to a non-portable electronicsystem. For example, earphones may be considered a portable electronicdevice whether they are operated wirelessly or through a wireconnection.

The convenience afforded by the portability of electronic devices hasfostered a huge industry. Smartphones, audio players, laptop computers,tablet computers, and ebook readers are all examples of portableelectronic devices.

Wearable Electronic Devices

A wearable electronic device is any portable electronic device that auser can carry without physically grasping, clutching, or otherwiseholding onto the device with their hands. For example, a wearableelectronic device may be attached or coupled to the user by a strap orstraps, a band or bands, a clip or clips, an adhesive, a pin and clasp,an article of clothing, tension or elastic support, an interference fit,an ergonomic form, and the like. Examples of wearable electronic devicesinclude digital wristwatches, electronic armbands, electronic rings,electronic ankle-bracelets or “anklets,” head-mounted electronic displayunits, hearing aids, and so on.

Some types of wearable electronic devices that have the electronicdisplays described above may include wearable heads-up displays. Awearable heads-up display is a head-mounted display that enables theuser to see displayed content but does not prevent the user from beingable to see their external environment. A typical head-mounted display(e.g., well-suited for virtual reality applications) may be opaque andprevent the user from seeing their external environment, whereas awearable heads-up display (e.g., well-suited for augmented realityapplications) may enable a user to see both real and virtual/projectedcontent at the same time. A wearable heads-up display is an electronicdevice that is worn on a user's head and, when so worn, secures at leastone display within a viewable field of at least one of the user's eyesat all times, regardless of the position or orientation of the user'shead, but this at least one display is either transparent or at aperiphery of the user's field of view so that the user is still able tosee their external environment. Examples of wearable heads-up displaysinclude: Google Glass®, Optinvent Ora®, Epson Moverio®, and SonyGlasstron®.

Text Entry

It is highly desirable to be able to input text to an electronic device,for example to provide control instructions, to write a message, or toprepare documents. However, one problem with wearable electronic devicesand portable electronic devices today is that the user is forced to useinefficient and/or unergonomic mechanisms for text entry. For example,modern smartphones attempt to mimic the traditional keyboard bydisplaying an on-screen keyboard. This on-screen keyboard typicallyoccupies nearly half of the screen space of the smartphone, and thussignificantly occludes displayed content. Further, many of the functionsof the keyboard, such as punctuation and numbers, are often relegated tosecondary keyboards, forcing the user to switch back-and-forth betweendifferent keyboards to achieve normal text input. Further still, suchon-screen keyboards typically have a form factor which causes users touse only two fingers to input text, significantly reducing efficiency ofthe keyboard. Additionally, on-screen keyboards are highly prone toinaccuracy and frequently detect a key different from the key which theuser intended to enter, which is highly inefficient and frustrating tothe user.

Text entry on wearable electronic devices suffers from even greaterproblems than those faced by smartphones. A smartwatch, for example, hasan even smaller screen than a smartphone, and thus faces the sameproblems described above regarding smartphones, but to a much greatermagnitude. This has led to many smartwatches simply having no mechanismfor text entry.

Wearable heads-up displays have even greater struggles for text entrythan the devices mentioned above. For example, wearable heads-updisplays do not typically employ touchscreens, and thus cannot use touchbased on-screen keyboards like in smartphones. Further, using anon-touch based on-screen keyboard, such as by having a user move acursor using a joystick or buttons on the wearable heads-up display isincredibly inefficient, forcing the user to scroll across an entirekeyboard for each individual key. Further still, using an additionaldevice, such as a portable keyboard connected to the wearable heads-updisplay, increases overall bulk of the system, and forces the user tocarry an additional component around, and to retrieve the additionalcomponent each time it is needed. This significantly hampers a number ofthe key advantages of using a wearable electronic device, such asmaintaining your hands free for other tasks, and reducing the number andweight of components that need to be carried around.

To some extent the above issues related to text entry have seensolutions in the form of speech-to-text entry, where the user speaks tothe device and the device converts the user's speech to text. However,these systems have many drawbacks, such as lack of privacy. If a userwishes to send a private message, but is in a public space, then it issimply not possible to send the speech-to-text message privately, sinceeveryone around the user will hear what is being said to the device.Further, these systems are also highly conspicuous, and draw attentionto the user in situations where it may be considered rude or sociallyinappropriate to communicate vocally through your device, such as on acrowded bus or train. Further still, these systems can be prone toinaccuracy, due to the difficulty involved in a computer processortrying to process human speech.

User-to-User Messaging

A key feature of portable electronic devices is the ability tocommunicate information between users of the devices, even when theusers are far apart. Presently, this is typically achieved withmessaging, where a user of one device sends a string of characters(e.g., text) and/or images (e.g., “emoji”) to another device as amessage. The advantages of communicating with such messages arenumerous. For example, the size of the message can be kept to a minimum,since text strings and Unicode emoji require only a small amount ofdata, and thus bandwidth usage can be kept small when using suchmessages. Additionally, message-type communications do not requireimmediate attention by the recipient, but instead can be reviewedanytime at the recipient's convenience. For at least these reasons, itis highly desirable to provide an effective means of enabling thecreation of messages for communication on electronic devices, andparticularly on portable or wearable electronic devices that havelimited means for text input.

Content Submission Messaging

Another feature of portable electronic devices is the ability tocommunicate information between a user of a device and a content hub,even when the users are far apart. For example, a user can submit areview to a restaurant website, or to a third-party review aggregationsite such as TripAdvisor® or Yelp®. As another example, a user cansubmit content to social media services such as Facebook®. Such contentsubmitted by the user can be considered as a “message”, and can containfor example text, emojis, images, video, audio, etc. It is highlydesirable to provide an effective means of enabling the creation of suchmessages on electronic devices, and particularly on portable or wearableelectronic devices that have limited means for text input.

BRIEF SUMMARY

According to a broad aspect, the description describes a processor-basedmethod of initiating a communication between a first system and a secondsystem, wherein the first system includes at least one processor, anoutput device communicatively coupled to the at least one processor, awireless communications module communicatively coupled to the at leastone processor, and a non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor, and wherein thenon-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to perform the method, may besummarized as including identifying, by the at least one processor, anenvironmental context parameter; generating, by the at least oneprocessor, at least one candidate message based on the environmentalcontext parameter; presenting, by the output device, the at least onecandidate message; receiving, by the at least one processor, a firstinput from a user of the first system, the first input indicative of aselection of one of the at least one candidate message as a selectedmessage; and transmitting, by the wireless communications module, theselected message.

The first system may include a wearable heads-up display (“WHUD”) havingat least one processor, a display communicatively coupled to the atleast one processor of the WHUD, a first wireless communications modulecommunicatively coupled to the at least one processor of the WHUD, and anon-transitory processor-readable storage medium communicatively coupledto the at least one processor of the WHUD; the first system may includea peripheral device having a second wireless communications module;identifying, by the at least one processor, an environmental contextparameter may include identifying, by the at least one processor of theWHUD, an environmental context parameter; generating, by the at leastone processor, at least one candidate message based on the environmentalcontext parameter may include generating, by the at least one processorof the WHUD, at least one candidate message based on the environmentalcontext parameter; presenting, by the output device, the at least onecandidate message may include displaying, by the display of the WHUD,the at least one candidate message; receiving, by the at least oneprocessor, a first input from a user of the first system may includereceiving, by the at least one processor of the WHUD, a first input froma user of the first system, the first input indicative of the selectionof one of the at least one candidate message as the selected message;and transmitting, by the wireless communications module, the selectedmessage may include: transmitting, by the first wireless communicationsmodule of the WHUD, the selected message; receiving, by the secondcommunications module of the peripheral device, the selected message;and transmitting, by the second wireless communications module of theperipheral device, the selected message.

The first system may include a wearable heads-up display (“WHUD”) havinga display; the first system may include a peripheral device having atleast one processor, a wireless communications module communicativelycoupled to the at least one processor of the peripheral device, and anon-transitory processor-readable storage medium communicatively coupledto the at least one processor of the peripheral device; identifying, bythe at least one processor, an environmental context parameter mayinclude identifying, by the at least one processor of the peripheraldevice, an environmental context parameter; generating, by the at leastone processor, at least one candidate message based on the environmentalcontext parameter may include generating, by the at least one processorof the peripheral device, at least one candidate message based on theenvironmental context parameter; presenting, by the output device, theat least one candidate message may include displaying, by the display ofthe WHUD, the at least one candidate message; receiving, by the at leastone processor, a first input from a user of the first system may includereceiving, by the at least one processor of the peripheral device, afirst input from a user of the first system, the first input indicativeof a selection of one of the at least one candidate message as theselected message; and transmitting, by the wireless communicationsmodule, the selected message may include transmitting, by the wirelesscommunications module of the peripheral device, the selected message.

The WHUD may further include a wireless receiver communicatively coupledto the display, the method further including: transmitting, by thewireless communications module of the peripheral device, the at leastone candidate message; and receiving, by the wireless receiver of theWHUD, the at least one candidate message before displaying, by thedisplay of the WHUD, the at least one candidate message. Identifying, bythe at least one processor, an environmental context parameter mayinclude identifying, by the at least one processor, an environmentalcontext parameter of the first system. Identifying, by the at least oneprocessor, an environmental context parameter may include identifying,by the at least one processor, and environmental context parameter of auser of the first system.

The first system may further include a clock communicatively coupled tothe at least one processor; identifying, by the at least one processor,an environmental context parameter may include identifying, by the atleast one processor, a time of day based on an output of the clock; andgenerating, by the at least one processor, at least one candidatemessage may include generating, by the at least one processor, at leastone greeting appropriate for the time of day.

The first system may further include a clock communicatively coupled tothe at least one processor; identifying, by the at least one processor,an environmental context parameter may include identifying, by the atleast one processor, a date based on an output of the clock; andgenerating, by the at least one processor, at least one candidatemessage may include generating, by the at least one processor, at leastone greeting appropriate for the date.

The first system may further include a clock communicatively coupled tothe at least one processor; identifying, by the at least one processor,an environmental context parameter may include identifying, by the atleast one processor, a time of day and a date based on an output of theclock and identifying at least one relevant calendar entry in a calendarbased on the time of day and the date; and generating, by the at leastone processor, at least one candidate message may include generating, bythe at least one processor, at least one candidate message based on theat least one relevant calendar entry.

The first system may further include at least one input sensorcommunicatively coupled to the at least one processor; identifying, bythe at least one processor, an environmental context parameter mayinclude identifying, by the at least one processor, an activityperformed by the user of the first system based on input sensor dataprovided by the at least one input sensor; and generating, by the atleast one processor, at least one candidate message may includegenerating, by the at least one processor, at least one candidatemessage based on the activity performed by the user of the first system.

The method may further include receiving, by a receiver of the firstsystem, input sensor data, wherein identifying, by the at least oneprocessor, an environmental context parameter may include identifying,by the at least one processor, an activity performed by the user of thefirst system based on the input sensor data; and generating, by the atleast one processor, at least one candidate message may includegenerating, by the at least one processor, at least one candidatemessage based on the activity performed by the user of the first system.

The first system may further include a location sensor communicativelycoupled to the at least one processor; identifying, by the at least oneprocessor, an environmental context parameter may include identifying,by the at least one processor, a location of the first system based onan output of the location sensor; and generating, by the at least oneprocessor, at least one candidate message may include generating, by theat least one processor, at least one candidate message based on thelocation of the first system.

The first system may further include at least one physical environmentsensor communicatively coupled to the at least one processor;identifying, by the at least one processor, an environmental contextparameter may include identifying, by the at least one processor, atleast a physical environment of the first system based on an output ofthe physical environment sensor; and generating, by the at least oneprocessor, at least one candidate message may include generating, by theat least one processor, at least one candidate message based on thephysical environment of the first system.

The method may further include determining, by the at least oneprocessor, at least one candidate recipient for the selected messagebased on at least one of a content of the selected message and theenvironmental context parameter; presenting, by the output device, theat least one candidate recipient for the selected message; andreceiving, by the at least one processor, a second input from the userof the first system, the second input indicative of a selection of oneof the at least one candidate recipient as a selected recipient, whereintransmitting, by the wireless communications module, the selectedmessage includes transmitting, by the wireless communications module,the selected message to the selected recipient.

The method may further include autonomously modifying, by the at leastone processor, the selected message based on the selected recipient.Receiving, by the at least one processor, a second input from the userof the first system, the second input indicative of a selection of oneof the at least one candidate recipient as a selected recipient may beperformed prior to receiving, by the at least one processor, a firstinput from a user of the first system, the first input indicative of aselection of one of the at least one candidate message as the selectedmessage.

The method may further include determining, by the at least oneprocessor, a communication vernacular based on a communication historybetween the user of the first system and the selected recipient, whereingenerating, by the at least one processor, at least one candidatemessage based on the environmental context parameter further includesgenerating, by the at least one processor, the at least one candidatemessage based on the communication vernacular.

The method may further include determining, by the at least oneprocessor, a relationship between the user of the first system and theselected recipient, wherein generating, by the at least one processor,at least one candidate message based on the environmental contextparameter further includes generating, by the at least one processor,the at least one candidate message based on the determined relationship.

The method may further include autonomously determining, by the at leastone processor, a recipient of the selected message based on at least oneof a content of the selected message and the environmental contextparameter.

The method may further include, prior to generating, by the at least oneprocessor, at least one candidate message: determining, by the at leastone processor, at least one candidate recipient based on theenvironmental context parameter; presenting, by the output device, theat least one candidate recipient; and receiving, by the at least oneprocessor, a second input from the user of the first system, the secondinput indicative of a selected recipient from the at least one candidaterecipient, wherein transmitting, by the wireless communications module,may include transmitting the selected message to the selected recipient.

Generating, by the at least one processor, at least one candidatemessage based on the environmental context parameter may includegenerating, by the at least one processor, at least two candidatemessages based on the environmental context parameter, and receiving, bythe at least one processor, a first input from a user of the firstsystem may include receiving, by the at least one processor, the firstinput from the user of the first system indicative of a selection of oneof the at least two candidate messages.

Generating, by the at least one processor, at least one candidatemessage may include generating, by the at least one processor, a firstportion of a message; generating, by the at least one processor, atleast two candidate second portions to complete the message based on atleast the first portion of the message; receiving, by the at least oneprocessor, a second input from a user of the first system, the secondinput indicative of a selected second portion from the at least twocandidate second portions, generating, by the at least one processor, acomplete message by combining the first portion and the selected secondportion.

In another broad aspect, the subject description describes a firstsystem which initiates a communication between the first system and asecond system which may be summarized as including at least oneprocessor; an output device communicatively coupled to the at least oneprocessor; a wireless communications module communicatively coupled tothe at least one processor; and a non-transitory processor-readablestorage medium communicatively coupled to the at least one processor;wherein the non-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to: identify, by the at least oneprocessor, an environmental context parameter; generate, by the at leastone processor, at least one candidate message based on the environmentalcontext parameter; cause a presentation, by the output device, of the atleast one candidate message; receive, by the at least one processor, afirst input from a user of the first system, the first input indicativeof a selected message from the at least one candidate message; and causea transmission, by the wireless communications module, of the selectedmessage.

The first system may further include a wearable heads-up display(“WHUD”) which may include the at least one processor, a displaycommunicatively coupled to the at least one processor, a first wirelesscommunications module communicatively coupled to the at least oneprocessor, and the non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; and a peripheraldevice comprising a second wireless communications module; wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message, cause the first system to:transmit, by the first wireless communications module of the WHUD, theselected message; receive, by the second wireless communications moduleof the peripheral device, the selected message; and transmit, by thesecond wireless communications module of the peripheral device, theselected message.

The first system may further include a wearable heads-up display(“WHUD”) comprising a display; and a peripheral device comprising the atleast one processor, the wireless communications module communicativelycoupled to the at least one processor, and the non-transitoryprocessor-readable storage medium communicatively coupled to the atleast one processor.

The WHUD may further include a wireless receiver communicatively coupledto the output device, the processor-executable instructions furthercomprising processor-executable instructions that, when executed by theat least one processor, cause the first system to: transmit, by thewireless communications module of the peripheral device, the at leastone candidate message; and receive, by the wireless receiver of theWHUD, the at least one candidate message. The environmental contextparameter may represent an environmental context of the first system.The environmental context parameter may represent an environmentalcontext of a user of the first system.

The first system may further include a clock communicatively coupled tothe at least one processor, wherein the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify a time of day based on an output of the clock; and theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to generate, by the at least oneprocessor, at least one candidate message cause the at least oneprocessor to generate at least one greeting appropriate for the time ofday.

The first system may further include a clock communicatively coupled tothe at least one processor, wherein the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify a date based on an output of the clock; and theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to generate, by the at least oneprocessor, at least one candidate message cause the at least oneprocessor to generate at least one greeting appropriate for the date.

The first system may further include a clock communicatively coupled tothe at least one processor, wherein the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify a time of day and a date based on an output of the clock andidentify at least one relevant calendar entry in a calendar based on thetime of day and the date; and the processor-executable instructionsthat, when executed by the at least one processor, cause the firstsystem to generate, by the at least one processor, at least onecandidate message cause the at least one processor to generate at leastone candidate message based on the at least one relevant calendar entry.

The first system may further include at least one input sensorcommunicatively coupled to the at least one processor, wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to identify, by the at least oneprocessor, an environmental context parameter cause the at least oneprocessor to identify an activity performed by the user of the firstsystem based on input sensor data provided by the at least one inputsensor; and the processor-executable instructions that, when executed bythe at least one processor, cause the first system to generate, by theat least one processor, at least one candidate message cause the atleast one processor to generate at least one candidate message based onthe activity performed by the user of the first system. The at least oneinput sensor may include at least one sensor selected from a groupconsisting of: an electromyography sensor, a mechanomyography sensor, anaccelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a globalpositioning system sensor, and a compass. The processor-executableinstructions that, when executed by the at least one processor, maycause the first system to identify, by the at least one processor, anenvironmental context parameter may cause the at least one processor toidentify an activity performed by the user of the first system based oninput sensor data received from a sensing device; and theprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to generate, by the at leastone processor, at least one candidate message may cause the at least oneprocessor to generate at least one candidate message based on theactivity performed by the user of the first system.

The first system may further include a location sensor communicativelycoupled to the at least one processor, wherein the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify a location of the first system based on an output of thelocation sensor; and the processor-executable instructions that, whenexecuted by the at least one processor, cause the first system togenerate, by the at least one processor, at least one candidate messagecause the at least one processor to generate at least one candidatemessage based on the location of the first system.

The first system may further include at least one physical environmentsensor communicatively coupled to the at least one processor, whereinthe processor-executable instructions that, when executed by the atleast one processor, cause the first system to identify, by the at leastone processor, an environmental context parameter cause the at least oneprocessor to identify at least a physical environment of the firstsystem based on an output of the physical environment sensor; and theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to generate, by the at least oneprocessor, at least one candidate message cause the at least oneprocessor to generate at least one candidate message based on thephysical environment of the first system. The at least one physicalenvironment sensor may include at least one sensor selected from a groupconsisting of: a camera and a microphone.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to: determine, by the at leastone processor, at least one candidate recipient for the selected messagebased on at least one of a content of the selected message and theenvironmental context parameter; present, by the output device, the atleast one candidate recipient for the selected message; and receive, bythe at least one processor, a second input from the user of the firstsystem, the second input indicative of a selected recipient from the atleast one candidate recipient, wherein the processor-executableinstructions that, when executed by the at least one processor, causethe first system to transmit, by the wireless communications module, theselected message cause the wireless communications module to transmitthe selected message to the selected recipient.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to modify, by the at least oneprocessor, the selected message based on the selected recipient.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to determine, by the at leastone processor, a communication vernacular based on a communicationhistory between the user of the first system and the selected recipient,wherein the processor-executable instructions that, when executed by theat least one processor, may cause the first system to generate, by theat least one processor, at least one candidate message cause the atleast one processor to generate at least one candidate message based onthe communication vernacular.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to determine, by the at leastone processor, a relationship between the user of the first system andthe selected recipient, wherein the processor-executable instructionsthat, when executed by the at least one processor, may cause the firstsystem to generate, by the at least one processor, at least onecandidate message may cause the at least one processor to generate atleast one candidate message based on the determined relationship. Thenon-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to, by the at least oneprocessor, autonomously determine a recipient of the selected messagebased on at least one of a content of the selected message and theenvironmental context parameter.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to: prior to generation, bythe at least one processor, of at least one candidate message:determine, by the at least one processor, at least one candidaterecipient based on the environmental context parameter; present, by theoutput device, the at least one candidate recipient; and receive, by theat least one processor, a second input from the user of the firstsystem, the second input indicative of a selected recipient from the atleast one candidate recipient, and the processor-executable instructionsthat, when executed by the at least one processor, cause the firstsystem to transmit, by the wireless communications module, the selectedmessage may cause the wireless communications module to transmit theselected message to the selected recipient.

The processor-executable instructions that, when executed by the atleast one processor, may cause the first system to generate, by the atleast one processor, at least one candidate message based on theenvironmental context parameter may cause the at least one processor togenerate at least two candidate messages based on the environmentalcontext parameter; and the processor-executable instructions that, whenexecuted by the at least one processor, may cause the first system toreceive, by the at least one processor, a first input from a user of thefirst system, may cause the at least one processor to receive a firstinput indicative of a selected message from the at least two candidatemessages.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to generate, by the at leastone processor, a first portion of a message; the processor-executableinstructions which may cause the first system to generate, by the atleast one processor, at least one candidate message may cause the atleast one processor to generate at least two candidate second portionsto complete the message based on at least the first portion of themessage; the processor-executable instructions which may cause the firstsystem to receive, by the at least one processor, a first input from auser of the first system may cause the at least one processor to receivea first input from a user of the first system indicative of a selectedsecond portion from the at least two candidate second portions; and thenon-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to generate, by the at leastone processor, a complete message by combining the first portion and theselected second portion.

In another broad aspect, the subject description describes aprocessor-based method of initiating a communication between a firstsystem and a second system, wherein the first system includes at leastone processor, an output device communicatively coupled to the at leastone processor, a wireless communications module communicatively coupledto the at least one processor, and a non-transitory processor-readablestorage medium communicatively coupled to the at least one processor,and wherein the non-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to perform the method, may besummarized as including generating, by the at least one processor, afirst portion of a message; presenting, by the output device, the firstportion of the message; generating, by the at least one processor, atleast two candidate second portions to complete the message based on atleast the first portion of the message; presenting, by the outputdevice, the at least two candidate second portions; receiving, by the atleast one processor, a first input from a user of the first system, thefirst input indicative of a selection of one of the at least twocandidate second portions as a selected second portion; generating, bythe at least one processor, a complete message by combining the firstportion and the selected second portion; and transmitting, by thewireless communications module, the complete message from the firstsystem.

The first system may include a wearable heads-up display (“WHUD”) havingat least one processor, a display communicatively coupled to the atleast one processor of the WHUD, a first wireless communications modulecommunicatively coupled to the at least one processor of the WHUD, and anon-transitory processor-readable storage medium communicatively coupledto the at least one processor of the WHUD; the first system may includea peripheral device having a second wireless communications module;generating, by the at least one processor, a first portion of a messagemay include generating, by the at least one processor of the WHUD, afirst portion of a message; presenting, by the output device, the firstportion of the message may include displaying, by the display of theWHUD, the first portion of the message; generating, by the at least oneprocessor, at least two candidate second portions to complete themessage based on at least the first portion of the message may includegenerating, by the at least one processor of the WHUD, at least twocandidate second portions to complete the message based on at least thefirst portion of the message; presenting, by the output device, the atleast two candidate second portions may include displaying, by thedisplay of the WHUD, the at least two candidate second portions;receiving, by the at least one processor, a first input from a user ofthe first system may include receiving, by the at least one processor ofthe WHUD, a first input from a user of the first system, the first inputindicative of a selection of one of the at least two candidate secondportions as the selected second portion; generating, by the at least oneprocessor, a complete message by combining the first portion and theselected second portion may include generating, by the at least oneprocessor of the WHUD, a complete message by combining the first portionand the selected second portion; and transmitting, by the wirelesscommunications module, the complete message from the first system mayinclude: transmitting, by the first wireless communications module ofthe WHUD, the complete message; receiving, by the second wirelesscommunications module of the peripheral device, the complete message;and transmitting, by the second wireless communications module of theperipheral device, the complete message.

The first system may include a wearable heads-up display (“WHUD”) havinga display; the first system may include a peripheral device having atleast one processor, a wireless communications module communicativelycoupled to the at least one processor of the peripheral device, and anon-transitory processor-readable storage medium communicatively coupledto the at least one processor of the peripheral device; generating, bythe at least one processor, a first portion of a message may includegenerating, by the at least one processor of the peripheral device, afirst portion of a message; presenting, by the output device, the firstportion of the message may include displaying, by the display of theWHUD, the first portion of the message; generating, by the at least oneprocessor, at least two candidate second portions to complete themessage based on at least the first portion of the message may includegenerating, by the at least one processor of the peripheral device, atleast two candidate second portions to complete the message based on atleast the first portion of the message; presenting, by the outputdevice, the at least two candidate second portions may includedisplaying, by the display of the WHUD, the at least two candidatesecond portions; receiving, by the at least one processor, a first inputfrom a user of the first system may include receiving, by the at leastone processor of the peripheral device, a first input from a user of thefirst system, the first input indicative of a selection of one of the atleast two candidate second portions as the selected second portion;generating, by the at least one processor, a complete message bycombining the first portion and the selected second portion may includegenerating, by the at least one processor of the peripheral device, acomplete message by combining the first portion and the selected secondportion; and transmitting, by the wireless communications module, theselected message may include transmitting, by the wirelesscommunications module of the peripheral device, the complete message.

The WHUD may further include a wireless receiver communicatively coupledto the display, and the method may further include transmitting, by thewireless communications module of the peripheral device, the firstportion of the message; receiving, by the wireless receiver of the WHUD,the first portion of the message before displaying, by the display ofthe WHUD, the first portion of the message; transmitting, by thewireless communications module of the peripheral device, the at leasttwo candidate second portions; and receiving, by the wireless receiverof the WHUD, the at least two candidate second portions beforedisplaying, by the display of the WHUD, the at least two candidatesecond portions. At least one of the at least two candidate secondportions may be a text string, and presenting, by the output device, theat least two candidate second portions may include presenting the textstring by the output device. At least one of the at least two candidatesecond portions may be a non-text image, and presenting, by the outputdevice, the at least two candidate second portions may includedisplaying the image by the output device.

The method may further include identifying, by the at least oneprocessor, an environmental context parameter, wherein generating, bythe at least one processor, a first portion of a message includesgenerating, by the at least one processor, the first portion of themessage based on the environmental context parameter, wherein theenvironmental context parameter represents an environmental contextincluding at least one context selected from a group consisting of: atime, a date, a calendar entry in a calendar, a location, a physicalenvironment, and an activity performed by the user. Generating, by theat least one processor, at least two candidate second portions tocomplete the message based on at least the first portion of the messagemay include generating, by the at least one processor, the at least twocandidate second portions based on the environmental context parameter,wherein the environmental context parameter represents an environmentalcontext including at least one context selected from a group consistingof: a time, a date, a calendar entry in a calendar, a location, aphysical environment, and an activity performed by the user.

The method may further include autonomously determining, by the at leastone processor, at least one recipient of the complete message based onat least one of a content of the complete message and the environmentalcontext parameter.

The method may further include determining, by the at least oneprocessor, at least one candidate recipient for the complete messagebased on at least one of a content of the complete message and theenvironmental context parameter; presenting, by the output device, theat least one candidate recipient for the complete message; andreceiving, by the at least one processor, a second input from the userof the first system, the second input indicative of a selected recipientfrom the at least one candidate recipient, wherein transmitting, by thewireless communications module, the complete message includestransmitting, by the wireless communications module, the completemessage to the selected recipient.

In another broad aspect, the subject description describes a firstsystem which initiates a communication between the first system and asecond system may be summarized as including at least one processor; anoutput device communicatively coupled to the at least one processor; awireless communications module communicatively coupled to the at leastone processor; and a non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; wherein thenon-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to: generate, by the at least oneprocessor, a first portion of a message; present, by the output device,the first portion of the message; generate, by the at least oneprocessor, at least two candidate second portions to complete themessage based on at least the first portion of the message; present, bythe output device, the at least two candidate second portions; receive,by the at least one processor, a first input from a user of the firstsystem, the first input indicative of a selection of one the at leasttwo candidate second portions as a selected second portion; generate, bythe at least one processor, a complete message by combining the firstportion and the selected second portion; and transmit, by the wirelesscommunications module, the complete message from the first system.

The first system may further include a wearable heads-up display(“WHUD”) which comprises the at least one processor, a displaycommunicatively coupled to the at least one processor, a first wirelesscommunications module communicatively coupled to the at least oneprocessor, and the non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; and a peripheraldevice comprising a second wireless communications module; wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message, cause the first system to:transmit, by the first wireless communications module of the WHUD, thecomplete message; receive, by the second wireless communications moduleof the peripheral device, the complete message; and transmit, by thesecond wireless communications module of the peripheral device, thecomplete message.

The first system may further include a wearable heads-up display(“WHUD”) comprising a display; and a peripheral device comprising the atleast one processor, the wireless communications module communicativelycoupled to the at least one processor, and the non-transitoryprocessor-readable storage medium communicatively coupled to the atleast one processor.

The WHUD may further include a wireless receiver communicatively coupledto the display, the processor-executable instructions further includingprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module of the peripheral device, the first portion of themessage; receive, by the wireless receiver of the WHUD, the firstportion of the message; transmit, by the wireless communications moduleof the peripheral device, the at least two candidate second portions;and receive by the wireless receiver of the WHUD, the at least twocandidate second portions. At least one of the at least two candidatesecond portions may be a text string, and the processor-executableinstructions that, when executed by the at least one processor, causethe first system to display, by the display, the at least two candidatesecond portions may cause the display to display the text string. Atleast one of the at least two candidate second portions may be anon-text image, and the processor-executable instructions that, whenexecuted by the at least one processor, cause the first system todisplay, by the display, the at least two candidate second portions maycause the display to display the image.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the at least one processor to identify anenvironmental context parameter; and the processor-executableinstructions that, when executed by the at least one processor, maycause the first system generate, by the at least one processor, a firstportion of a message may cause the at least one processor to generatethe first portion of the message based on the environmental contextparameter, wherein the environmental context parameter represents anenvironmental context including at least one context selected from agroup consisting of: a time, a date, a calendar entry in a calendar, alocation, a physical environment, and an activity performed by the user.The processor-executable instructions that, when executed by the atleast one processor, may cause the first system to generate, by the atleast one processor, at least two candidate second portions to completethe message based on at least the first portion of the message may causethe at least one processor to generate the at least two candidate secondportions based on the environmental context parameter, wherein theenvironmental context parameter represents an environmental contextincluding at least one context selected from a group consisting of: atime, a date, a calendar entry in a calendar, a location, a physicalenvironment, and an activity performed by the user.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the at least one processor to autonomouslydetermine at least one recipient of the complete message based on atleast one of a content of the complete message and the environmentalcontext parameter.

The non-transitory processor-readable storage medium may store furtherprocessor-executable instructions that, when executed by the at leastone processor, may cause the first system to determine, by the at leastone processor, at least one candidate recipient for the complete messagebased on at least one of a content of the complete message and theenvironmental context parameter; present, by the output device, the atleast one candidate recipient for the complete message; and receive, bythe at least one processor, a second input from the user of the firstsystem, the second input indicative of a selection of one of the atleast one candidate recipient as a selected recipient, wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the complete message cause the wirelesscommunications module to transmit the complete message to the selectedrecipient.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings, identical reference numbers identify similar elementsor acts. The sizes and relative positions of elements in the drawingsare not necessarily drawn to scale. For example, the shapes of variouselements and angles are not necessarily drawn to scale, and some ofthese elements are arbitrarily enlarged and positioned to improvedrawing legibility. Further, the particular shapes of the elements asdrawn are not necessarily intended to convey any information regardingthe actual shape of the particular elements, and have been solelyselected for ease of recognition in the drawings.

FIGS. 1A, 1B, and 1C are schematic diagrams showing exemplary systemswhich can exchange messages in the subject systems, devices, andmethods.

FIG. 2A is an isometric view of a wearable heads-up display on whichmessages can be generated and transmitted, according to at least oneillustrated implementation.

FIGS. 2B, 2C, and 2D are front plan views of handheld or laptopprocessor-based devices on which messages can be generated andtransmitted, according to three respective illustrated implementations.

FIG. 3 is a schematic diagram that illustrates an example of a systemwhich is used for reference to describe the subject systems, devices,and methods, according to at least one illustrated implementation.

FIG. 4 is a flow diagram that illustrates an exemplary method forgenerating a message, according to at least one illustratedimplementation.

FIGS. 5A, 5B, and 6 are representations of user interfaces which displayor present image representations including user selectable messages to auser in response to execution of the method of FIG. 4 .

FIG. 7 is a flow diagram that illustrates an exemplary method forgenerating a message and generating candidate recipients for themessage, according to at least one illustrated implementation.

FIGS. 8A, 8B, 8C, 9A, and 9B are representations of user interfaceswhich present image representations including user selectable messagesto a user in response to execution of the method of FIG. 7 .

FIGS. 10A, 10B, and 10C are representations of user interfaces whichpresent image representations including user selectable messages inresponse to execution of the method of FIG. 7 with the acts beingperformed in a different order.

FIG. 11 is a flow diagram that illustrates an exemplary method forgenerating a message by combining first and second portions of amessage, according to at least one illustrated implementation.

FIGS. 12A, 12B and 12C are representations of user interfaces whichdisplay or present image representations including user selectablemessages in response to execution of the method of FIG. 11 .

FIG. 13 is a flow diagram that illustrates an exemplary method forgenerating a message by combining first and second portions of amessage, and determining at least one candidate recipient for themessage, according to at least one illustrated implementation.

FIGS. 14A, 14B, 14C, 14D, 15A, 15B, 15C, and 15D are representations ofuser interfaces which present image representations including userselectable messages in response to execution of the method of FIG. 13 .

FIG. 16 is a flow diagram that illustrates an exemplary method forgenerating a message based on captured visual data according to at leastone illustrated implementation.

FIG. 17 is a flow diagram that illustrates an exemplary method forgenerating a message based on captured visual data, and determining atleast one candidate recipient for the message, according to at least oneimplementation.

FIGS. 18A, 18B, 18C, 18D, 19A, 19B, 19C, 19D, 20A, 20B, 20C, 20D, 21A,21B, 21C, 21D, 21E, 22A, 22B, 22C, 22D, 23A, 23B, 23C, 23D, 24A, 24B,24C, 24D, 25A, 25B, 25C, 26A, 26B, and 26C are representations of userinterfaces which present image representations including user selectablemessages in at least nine illustrated implementations of the methods ofFIG. 16 and FIG. 17 .

FIGS. 27,28, and 29 are exemplary scenes as seen from a camera, whichcan be analyzed to generate messages according to the methods of FIGS.16 and 17 .

DETAILED DESCRIPTION

In the following description, certain specific details are set forth inorder to provide a thorough understanding of various disclosedimplementations or embodiments. However, one skilled in the relevant artwill recognize that implementations or embodiments may be practicedwithout one or more of these specific details, or with other methods,components, materials, etc. In other instances, well-known structuresassociated with portable electronic devices and head-worn devices, havenot been shown or described in detail to avoid unnecessarily obscuringdescriptions of the implementations or the embodiments.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, that is as “including, but not limited to.”

Reference throughout this specification to “one implementation” or “animplementation” or “one embodiment” or “an embodiment” means that aparticular feature, structures, or characteristics may be combined inany suitable manner in one or more implementations or one or moreembodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its broadest sense, that is as meaning “and/or”unless the content clearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theimplementations or embodiments.

The various embodiments described herein provide systems, devices, andmethods for generating messages and are particularly well-suited for usein portable electronic devices or wearable electronic devices, but arealso fully applicable in non-portable electronic devices as well.

FIG. 1A shows an exemplary first system 100 and second system 190between which messages may be exchanged in accordance with the presentsystems, devices, and methods. First system 100 generates and sends amessage to at least a second system 190. The message can be sent to asingle system 190 but may also be sent to a plurality of systems 190, orvia a plurality of systems (e.g., intermediary servers between a senderand intended recipient(s)). System(s) 190 may include any appropriatesystem(s) capable of receiving a message, including but not limited to aphone, smartphone, laptop, desktop computer, tablet, smartwatch,smartglasses, personal digital assistant, or any combination thereof.System(s) 190 could also include a system identical to the first system100. In the example of FIG. 1A, system 100 includes a wearable heads-updisplay (“WHUD”) 110 and a control ring 120. The WHUD 110 includes anoutput device 101, on which content can be displayed to a user of theWHUD 110 when worn on the head of the user. Though not illustrated inthe FIG. 1A, WHUD 110 also includes at least one processor, anon-transitory processor-readable storage medium, and a wirelesscommunications module which can include a wireless receiver, a wirelesstransmitter, and/or a wireless transceiver. WHUD 110 could for examplebe one of the WHUDs disclosed in U.S. Non-Provisional patent applicationSer. No. 15/145,576, U.S. Non-Provisional patent application Ser. No.15/167,458, or U.S. Non-Provisional patent application Ser. No.15/046,254, all of which are incorporated herein by reference in theirentirety, and which further includes various processor-executableinstructions described herein for performing communications. Controlring 120 includes a control pad 121 which can receive a user input.Further, although not illustrated in the FIG. 1A, control ring 120 alsoincludes a wireless communications module, and may optionally include atleast one processor and a non-transitory processor-readable storagemedium. Control ring 120 could for example be one of the rings disclosedin U.S. Provisional Patent Application Ser. No. 62/607,816, U.S.Provisional Patent Application Ser. No. 62/607,819, or U.S. ProvisionalPatent Application Ser. No. 62/671,248, all of which are incorporatedherein by reference in their entirety.

Control ring 120 in use is worn on a finger of a user, and the user canprovide input to the WHUD 110 via the control ring 120. In the case ofFIG. 1A, the control pad 121 is a pressable joystick or rocker switch,which allows the user to input directional commands by applying force,with or without haptic feedback, for example by tilting the joystick, aswell as a confirm command by pressing the joystick inwards (e.g.,depressing). Control ring 120 communicates the user input to the WHUD110 by the wireless communications module on the control ring 120. TheWHUD 110 receives the user input with the wireless receiver thereon.Control ring 120 advantageously provides a convenient input mechanismwhich can be used quickly and inconspicuously.

One skilled in the art will recognize that the above hardwaredescription is merely an exemplary hardware arrangement, and manydifferent systems could be employed within the context of the presentdisclosure. For example, control ring 120 and WHUD 110 do notnecessarily need to communicate wirelessly, but instead control ring 120could communicate with the WHUD 110 via a wired connection. Further,control ring 120 is not limited to a ring worn a finger of the user, butcould also be any other type of control device, such as for example aband worn on a user's arm or a control pad held in a user's hand.Further still, control pad 121 could be replaced by any appropriateinput mechanism known in the art, such as tactile buttons or atouch-sensitive trackpad, for example.

FIG. 1B illustrates a system 130 which is an alternative to the system100 in FIG. 1A. System 130 includes a WHUD 140, a control ring 150, anda peripheral device 160. WHUD 140 can include all of the same componentsas WHUD 110, and the discussion above regarding WHUD 110 fully appliesto WHUD 140. Control ring 150 can include all of the same components ascontrol ring 120, and the discussion above regarding control ring 120fully applies to control ring 150. Peripheral device 160 can include atleast a wireless communications module. Peripheral device 160 can alsooptionally include a wireless receiver, at least one processor, anon-transient processor-readable storage medium, an output device, and auser input mechanism. The purpose of peripheral device 160 is to providean additional device for performing some of the functionality that wouldotherwise be performed by the WHUD, such that the WHUD 140 can belighter and/or the system 130 as a whole can be more powerful. As anexample, peripheral device 160 could be a typical smartphone, whichincludes at least a processor and long-range communication hardware suchas 2G, 3G, 4G, LTE, and/or any other suitable communication hardware.Instead of putting long-range wireless communication hardware on WHUD140, the WHUD 140 can instead use short range communication such asBluetooth®, Zigbee®, or similar to communicate with a user's smartphone,which in turn can perform long-range wireless communication with secondsystem 190. This arrangement is advantageous in that a single set ofshort-range wireless communication hardware can be included on the WHUD140 to communicate with both the control ring 150 and the peripheraldevice 160, thus reducing the bulk and weight of WHUD 140. Additionally,peripheral device 160 can be used to implement more functionality thanjust communication. For example, at least some of the processingperformed by the system 130 can be performed by the peripheral device160 instead of the WHUD 140. As an example, the WHUD 140 could includeminimal hardware, which includes only the display 141, hardware requiredto control the display 141, and wireless communication hardware forcommunicating with the peripheral device 160. The peripheral device 160could process and render image data to send to the WHUD 140, such thatthe WHUD 140 need only display the pre-rendered data. Such a systemwould advantageously reduce the amount of hardware which is required onthe WHUD, thus reducing bulk, weight, and power consumption of the WHUD.The reduced power consumption also allows for the WHUD to use a smallerbattery, further reducing bulk and weight.

One skilled in the art will appreciate that control ring 150 does notnecessarily need to communicate with WHUD 140. Specifically, in theimplementation where the peripheral device 160 processes and rendersimage data to send to WHUD 140 for display, control ring 150 could sendinputs from a user to the peripheral device 160 instead of to WHUD 140.Peripheral device 160 may process the received inputs and alter therendering of image data to be sent to WHUD 140 based on the receivedinputs if needed.

Although FIG. 1B shows the control ring 150 and peripheral device 160 asbeing separate devices, one skilled in the art will appreciate that itis within the scope of the present systems, devices, and methods for thecontrol ring 150 and peripheral device 160 to be coupled together oreven to be the same device. For example, the control ring 150 could beimplemented as a band worn on a user's forearm during use, and theperipheral device 160 could also be attached to the same band.Alternatively, in addition to the previously described components, thecontrol ring 150 could include at least one processor, a non-transitoryprocessor-readable storage medium, and wireless communication hardware,and the control ring 150 could perform much of the processing instead ofthe WHUD 140, and/or could perform the long-range communication insteadof the WHUD 140, such that the control ring 150 effectively serves asthe peripheral device 160.

One example of a system in which a peripheral device performs much ofthe processing for a WHUD is discussed in U.S. patent application Ser.No. 14/704,663 (Publication No. US 2015/0325202), which is incorporatedherein by reference in its entirety.

As another implementation to minimize hardware components carried by aWHUD, the present systems, devices, and method also allow for processingto be performed by at least one, possibly many, processors to performprocessing for the WHUD over a computer network, such as a “cloud”environment. As an example, any of the WHUD 110, control ring 120, WHUD140, control ring 150, and peripheral device 160 could communicate withat least one remote processor, such as a user's home desktop computer ora centralized server which handles many devices, via the Internet, suchthat the at least one remote processor performs processing for thesystems 100 and 130. The results of the processing can be sent back tothe systems 100 and 130, where the results can be displayed on thedisplays 111 or 141, or be further processed by the at least oneprocessor of the WHUDs or the peripheral device 160.

FIG. 1C is a schematic view of an exemplary first system 170, whichincludes many hardware components. The hardware components illustratedin FIG. 1C and discussed below, could also be implemented in firstsystem 100 and first system 130 discussed above, as well as any of theother systems discussed herein, such as system 300 discussed below. Oneskilled in the art will appreciate that each of the illustratedcomponents in FIG. 1C is not required to achieve the present systems,devices, and methods, but rather that the components illustrated in FIG.1C are exemplary components which can be implemented in the presentsystems, devices, and methods.

First system 170 as shown in FIG. 1C includes multiple output deviceswhich can be used to present information to a user. These output devicescan include, for example, display 175, speaker 176, haptic outputinterface 177, and any other output device as appropriate for aparticular application. A WHUD in accordance with the present systems,devices, and methods can optionally include a plurality, all, or acombination of these output devices as shown in FIG. 1C, but onlyrequires only a single output device, such as a single display 175, asingle speaker 176, or a single haptic output interface 177. Notably,although the term “Wearable heads-up display” implies that a devicewhich includes a display, similar head-wearable devices exist which donot require a display, but instead present output to a user by aspeaker. Further, a first system in accordance with the present systems,devices, and methods can also include a plurality of any or all of thedescribed output devices, such as a plurality of displays 175, aplurality of speakers 176, and/or a plurality of haptic outputinterfaces 177.

First system 170 as shown in FIG. 1C includes multiple sensors, whichcan be used to capture input which can indicate an environmental contextof the user, and which can be used to capture input from the user whichprovides instructions to the first system 170. These sensors can includemicrophone 181, camera 182, inertial measurement unit (“IMU”) 183,eye-tracking system 184, proximity sensors 185, location sensor 186, andany other sensor as appropriate for a particular application. A WHUD inaccordance with the present systems, devices, and methods can optionallyinclude a plurality, all, or a combination of these sensors as shown inFIG. 1 , but only requires a single sensor, such as a single microphone181, a single camera 182, a single inertial measurement unit 183, asingle eye-tracking system 184, a single proximity sensor 185, or asingle location sensor 186. Further, a WHUD in accordance with thepresent systems, devices, and methods can also include a plurality ofany or all of the described sensors, such as a plurality of microphones181, a plurality of cameras 182, a plurality of inertial measurementunits 183, a plurality of eye-tracking systems 184, a plurality ofproximity sensors 185, and/or a plurality of location sensor 186. Sincethe above described sensors can be used to capture and measureinformation indicating environmental context of a user of the firstsystem, throughout this specification the term “user context sensor” canrefer to any of the above described sensors, or any other sensors whichcan be carried by a system in accordance with the present invention, asappropriate for a given application. However, one skilled in the artwill appreciate that the above described sensors can be more than just“user context sensors”, in that the above described sensors can performfunctions beyond just capturing user context, such as capturing userinstructions, for example.

WHUD 170 as shown in FIG. 1C also includes at least one processor 171and a non-transitory processor-readable storage medium 172communicatively coupled to the at least one processor 171. The at leastone processor 171 can be any suitable component which can executeinstructions or logic, including but not limited to a micro-controller,microprocessor, multi-core processor, integrated-circuit, ASIC, FGPA,programmable logic device, or any appropriate combination of thesecomponents. The non-transitory processor-readable storage medium 172 canbe any suitable component which can store instructions, logic, orprograms, including but not limited to non-volatile or volatile memory,read only memory (ROM), random access memory (RAM), FLASH memory,registers, magnetic hard disk, optical disk, or any combination of thesecomponents. The non-transitory processor-readable storage medium 172 canstore processor executable instructions which, when executed by the atleast one processor 171, cause the first system 170 to implement thepresent systems, devices, and methods. Further, each of the outputdevices and sensors can be communicatively coupled to the at least oneprocessor 171. That is, the at least one processor 171 can becommunicatively coupled to at least display 175, speaker 176, hapticoutput interface 177, microphone 181, camera 182, inertial measurementunit 183, eye tracking system 184, proximity sensors 185, and locationsensor 186.

Optionally, first system 170 can include wireless communicationhardware, such as a wireless receiver, a wireless transmitter, and/or awireless transceiver, for communicating with peripheral devices and/orexternal servers and content providers. FIG. 1C illustrates an exemplarywireless communications module 173.

Certain elements of FIG. 1C are drawn in dashed lines, to indicate thatthese devices or structures are not normally visible at the currentperspective of first system 170 as shown in FIG. 1C, because the dashedelements are enclosed within a housing of the first system 170, or areoccluded by other features such as the support structure or housing offirst system 170. However, one skilled in the art will appreciate thatthese devices or structures do not necessarily have to be enclosed inthe housing or behind other features of first system 170, but can becarried external to the housing or partially external as appropriate fora given application. Further, one skilled in the art will appreciatethat although the output devices and sensors are shown at certainlocations of the support structure of first system 170, one skilled inthe art will appreciate that any of the output devices and sensors canbe relocated to any location on the support structure as appropriate fora particular application.

One skilled in the art will also appreciate that systems 100 130, and170 could be replaced by any appropriate systems having at least oneprocessor, a non-transitory processor-readable storage medium, awireless communications module, an output device, and a user inputmechanism. In this regard, FIGS. 2A-2D illustrate several alternativehardware arrangements for a first system which could be used instead ofthe hardware arrangement of first system 100 shown in FIG. 1A, thehardware arrangement of first system 130 shown in FIG. 1B, or thehardware arrangement of first system 170 shown in FIG. 1C.

FIG. 2A shows a WHUD 200 which includes a display 201 and input buttons202. The placement of the input buttons 202 directly on the WHUD 200obviate the need for wireless or wired communications between a separatecontrol device. The input buttons 202 could be replaced by anyappropriate input mechanism, such as a touch-sensitive trackpad orjoystick for example.

FIG. 2B shows a portable electronic device 210 which includes a display211 and input buttons 212. The user can interact with the portableelectronic device 210 using the input buttons 212. Portable electronicdevice 210 could be a device carried in the hand of a user, or could beworn on the user's body in any number of ways. For example, portableelectronic device 210 could be strapped to a user's wrist, thusfunctioning as a smart watch.

FIG. 2C shows a smartphone 220 which includes a touchscreen display 221.The user of the smartphone 220 can provide text input using the keyboard223 displayed on the touchscreen display 221. Optionally, the smartphone220 may also include input buttons 222.

FIG. 2D shows a laptop computer 230 which includes a display 231 andkeyboard 232. The user can interact and provide text to the laptop viathe keyboard 232, among other possible inputs such as a trackpad orperipheral mouse.

Although not specifically denoted in the above figures, the systems ofFIGS. 2A-2D each include at least one processor, a non-transitoryprocessor-readable storage medium, and a wireless communications module.Further, features illustrated and described with reference to FIGS. 1A,1B, and 1C, such as the user input mechanisms, output devices, andsensors, could be implemented in any of the devices of FIGS. 2A-2D.

FIG. 3 illustrates an exemplary first system 300 that will be referencedwhen describing the illustrative implementations of methods, systems,and devices below. FIG. 3 shows a system 300, which includes at leastone processor 301, a non-transitory processor-readable storage medium302, a wireless communications module 303, at least one output device304, a user input mechanism 305, and optionally one or more sensors 306.Output device 304 could for example comprise a display device, an audiooutput device, or both. The inclusion of any sensors 306 is optional,and could include for example any of the sensors discussed withreference to FIG. 1C, such as microphone 181, camera 182, inertialmeasurement unit (“IMU”) 183, eye-tracking system 184, proximity sensors185, location sensor 186, and any other sensor as appropriate for aparticular application. Further, user input mechanism 305 could be asensor of the one or more sensors 306. As an example, user inputmechanism 305 could comprise a microphone which can receive instructionsfrom a user. Alternatively or additionally, user input mechanism 305could be a dedicated input device, such as a touchpad or buttons. Thesystem 300 in FIG. 3 is representative of any of the exemplary hardwarearrangements illustrated in FIGS. 1A, 1B, 1C and 2A-2D, as well as anyother suitable hardware arrangements including at least one processor, anon-transitory processor-readable storage medium, a wirelesscommunications module, an output device, and a user input mechanism. Theat least one processor can be any suitable component which can executeinstructions or logic, including but not limited to a micro-controller,microprocessor, multi-core processor, integrated-circuit, ASIC, FGPA,programmable logic device, or any appropriate combination of thesecomponents. The non-transitory processor-readable storage medium can beany suitable component which can store instructions, logic, or programs,including but not limited to non-volatile or volatile memory, read onlymemory (ROM), random access memory (RAM), FLASH memory, registers,magnetic hard disk, optical disk, or any combination of thesecomponents. The wireless communications module can be any suitablecomponent which can transmit signals wirelessly for reception by otherdevice. The wireless communications module could also be for example aradio or a transceiver, capable of receiving signals in addition tosending signals.

Although the term “system 300” used hereinafter can refer to a singlephysical device where all the components are included in the samehousing, “system 300” can also refer to systems which have multiplephysically separate components housed separately. Further, theconnections shown between components of system 300 can be either wiredor wireless connections. For example, even though system 300 is shown asincluding user input mechanism 305, this still encompasses hardwarearrangements like that shown in FIG. 1A, in which the user inputmechanism is a control ring 120 that is a physically separate piece ofhardware from the WHUD 110. Similarly, system 300 is also representativeof hardware arrangements like those shown in FIG. 1B. For example, atleast one processor 301, non-transitory processor-readable storagemedium 302, output device 304, user input mechanism 305, and sensors 306can all be carried by a WHUD, but wireless communications module 303 canbe carried on a peripheral device such that communication betweenwireless communications module 303 and at least one processor 301 iswireless (such as Bluetooth®, Zigbee®, or similar), such that at leastone processor 301 uses wireless communications module 303 carried on adifferent device as a gateway to a larger network, such as the Internet,As another example, system 300 is also representative of alternativeimplementations of FIG. 1B such as implementations in which the at leastone processor 301 can be located on a different piece of physicalhardware from output device 304. Particularly, at least one processor301, non-transitory processor-readable storage medium 302, and userinput mechanism 305 could all be carried on a band which is wearable ona user's arm, and communicate with a WHUD which in use is carried on thehead of a user and includes output device 304. Additionally, althoughthe user input mechanism 305 is shown as a plurality of directionalbuttons and a confirm button, one skilled in the art will appreciatethat any appropriate user input mechanism can be used, including atouchscreen, a touch-sensitive trackpad, a track wheel, switches, and/ora gesture detection system as described in, for example, USNon-Provisional patent application Ser. No. 14/520,081, U.S.Non-Provisional patent application Ser. No. 14/567,826, or U.S.Non-Provisional patent application Ser. No. 14/737,081, all of which areincorporated by reference herein in their entirety.

FIG. 4 is a flowchart which illustrates an exemplary method 400 inaccordance with the present disclosure. Although method 400 is describedwith reference to system 300 shown in FIG. 3 , method 400 is equallyapplicable to any hardware arrangement having at least one processor, anon-transitory processor-readable storage medium, a wirelesscommunications module, an output device, and a user input mechanism.Method 400 can be embodied as a set of processor-executable instructionsstored on the non-transitory processor-readable storage medium 302,which can be executed by the at least one processor 301. Method 400includes 5 acts 401, 402, 403, 404, and 405, though those of skill inthe art will appreciate that in alternative implementations certain actsmay be omitted and/or additional acts may be added. Those of skill inthe art will also appreciate that the illustrated order of the acts isshown for exemplary purposes only and may change in alternativeimplementations.

At 401, the at least one processor 301 identifies an environmentalcontext parameter. As is discussed in more detail below, anenvironmental context parameter refers to data which representsenvironmental context of system 300 or of a user of system 300, whichmay have relevance to the content included in a message and/or relevanceto selection of a recipient or recipients of a message. At 402, the atleast one processor 301 generates at least one candidate message basedon the environmental context parameter. At 403, the output device 304presents the at least one candidate message to the user. At 404, the atleast one processor 301 receives an input from the user indicative of aselected message from the at least one candidate message. At 405, thewireless communications module 303 transmits the selected message fromthe first system, to be received by one or a plurality of other systemsor devices. Each of these acts is discussed in more detail below.

As discussed above regarding FIG. 1B, although it is possible to performall of the acts of method 1300 on a single device, it is also possibleto remove some of the hardware from the device (such as a WHUD), andutilize a peripheral device comprising said hardware instead, in orderto reduce bulk in the display device. In the case of method 400, acts401, 402, and 404 could be performed by a processor of a WHUD, act 403could be performed by an output device of the WHUD, and act 405 could beperformed by a wireless communications module of a peripheral devicecommunicatively coupled to the WHUD, such that the processing andpresenting acts are performed on the WHUD, and the peripheral device isused as a wireless transmission gateway. Alternatively, acts 401, 402,and 404 could be performed by at least one processor on a peripheraldevice, act 403 could be performed by an output device on a WHUDcommunicatively coupled to the peripheral device, and act 405 could beperformed by a wireless communications module on the peripheral device,such that the peripheral device performs processing and transmission,whereas the WHUD performs displaying.

Identifying an environmental context parameter and generating at leastone candidate message, such as in acts 401 and 402 of method 400, isdiscussed in more detail below. As mentioned above, an environmentalcontext parameter refers to data which represents environmental contextof system 300 or of a user of system 300, which may have relevance tothe content included in a message and/or relevance to selection of arecipient or recipients of a message.

For example, system 300 can include a clock, and the environmentalcontext parameter can include a current time of day as identified by atleast one processor 301 based on an output of the clock. The at leastone processor 301 may use the identified current time of day to generatea message which includes a greeting appropriate for a time of day. Forexample, the at least one processor 301 could generate a message whichsays “Good morning” if the clock outputs a current time which is before11:00 AM. As another example, the at least one processor 301 couldgenerate a message that says “Free for lunch?” if the current timeoutput by the clock is between 10:00 AM and 12:00 PM.

The environmental context can also include a current date as identifiedby the at least one processor 301 based on the output of the clock. Theat least one processor 301 may use the identified current date togenerate a message which includes a greeting appropriate for the currentdate. For example, the at least one processor 301 could generate amessage that says “Happy New Year!” if the current date output by theclock is January 1. As another example, the at least one processor 301could generate a message that says “Looking forward to the weekend!” ifthe current date output by the clock corresponds to a Friday.

Identifying the environmental context parameter may also include the atleast one processor 301 identifying a location of system 300 based onlocation data output by a location sensor on system 300. The at leastone processor 301 may then generate at least one candidate message basedon the identified location. For example, if the at least one processor301 identifies that the user of system 300 is at a grocery store, the atleast one processor 301 may generate a message like “I'm at the grocerystore, need anything?”. As another example, if the at least oneprocessor 301 identifies that the user was stationary at a restaurantfor a period of time, the at least one processor 301 may generate amessage which incorporates information about the restaurant. Forexample, if the user was stationary at a restaurant called “Yum Food”for a period of over 20 minutes, the at least one processor 301 couldgenerate a message like “I just ate at Yum Food, it was delicious!” Theat least one processor 301 identifying the location of the user mayinvolve the at least one processor 301 comparing map data to coordinatesoutput by the location sensor of system 300 (for example GPS coordinatesor coordinates determined by cell-phone tower triangulation, BLUETOOTH®(BT) beacons), in order to determine a venue or venues which match theposition indicated by the coordinates output by the location sensor ofsystem 300. Further, the at least one processor 301 can also account forarrival time at a venue and departure time from a venue. For example, ifthe user enters a movie theater at 2:00 PM, and leaves the movie theatreat 4:30 PM, the at least one processor 301 can generate a message like“I really enjoyed the movie.” The at least one processor 301 may alsoretrieve additional information from other sources in order to generatea more detailed message. In the above example, the at least oneprocessor could use the Internet to look up what movies were playing atthe movie theatre between the user's arrival time and departure time,and generate a message including this information. For example, if“Great Movie 5” was playing from 2:15 PM to 4:15 PM, the at least oneprocessor 301 could generate a message like “I just saw Great Movie 5,it was greater than the last one!” One skilled in the art willappreciate that the above time frames are merely exemplary, and could beany appropriate times as required by a specific application. Thisprocess can be performed completely by the at least one processor 301,or can be performed by a remote server, or may implement a third partyservice such as Pilgrim by Foursquare®. The map data can be stored onthe non-transitory processor-readable storage medium 302, or may bestored on a remote server accessible to the system 300. Alternatively,if the system 300 uses a remote service to identify the location of theuser, the map data may not need to be accessible to the system 300 atall, and identification of the location of the user may happen remotely.

Additionally, the environmental context can also include data stored ina calendar accessed by system 300. The calendar can be stored on thenon-transitory processor-readable storage medium 302 of system 300, ormay be stored elsewhere, such as on a remote server, which is accessedby system 300 over a network or the internet. After identifying thecurrent time and date output by the clock, the at least one processor301 may review the calendar for any relevant entries based on thecurrent time and date, and generate a message based on at least onerelevant calendar entry. The relevancy of a calendar entry can beassessed based on at least the start time of the entry, end time of theentry, name or description of the entry, priority level of the entry,attendees related to the entry, or any other suitable criteria. Forexample, the at least one processor 301 may identify that the user has ameeting scheduled in the calendar starting in 10 minutes, and maygenerate helpful messages like “Running late, start without me”, or “Seeyou in 10 minutes”. As another example, the at least one processor 301may identify that the user is currently in a meeting that is scheduledto end in 5 minutes, and may generate messages like “Wrapping up, I'llsee you soon” or “We still have a lot to do, I will probably be late”.

The calendar can include both one-time and recurring meetings, events,or tasks. The calendar can be populated manually, for example by theuser manually inputting an event, or can be populated by machinelearning. For example, if the user sends a message like “Picking up thekids” at 3:30 PM one day from a given location, the at least oneprocessor 301 may associate that location and time with picking up thekids. Subsequently, the next time the user accesses system 300 near thesame location at a similar time, the at least one processor 301 mayautonomously generate a message like “Picking up the kids”. As a furtherexample, the at least one processor 301 could generate a message thatsays “On my way home” if the time is close to the time at which the usertypically departs from work. This typical departure time could have beenmanually input by the user during configuration of system 300, or system300 could use machine learning based on a location sensor (e.g., GPSreceiver) in system 300 and/or messaging history, to learn the user'stravel patterns over time and determine when the user typically goeshome.

The at least one processor 301 may also generate candidate messagesbased on social norms. For example, if a clock of system 300 outputsthat it is currently Friday afternoon, the at least one processor 301may generate candidate messages like “It's Friday, let's go grab somebeers!”. Data relating to social norms can be stored on a non-transientprocessor-readable storage medium 302 of system 300, or can be stored ona remote server accessible to system 300.

Identifying the environmental context parameter may also include the atleast one processor 301 identifying the weather relevant to the user.For example, the at least one processor 301 may access an online weatherforecast for the current time and date as output by the clock of system300, and for the region surrounding the location of system 300 as outputby a location sensor of system 300. The at least one processor 301 maythen generate at least one candidate message based on the currentweather. For example, as the user is leaving their house in the morning,the at least one processor 301 may determine that the weather isparticularly cold that day, and generate a candidate message like “It'sreally cold today, make sure to dress warm” to send to the spouse orchildren of the user. Further, identifying the weather does not need tobe limited to the current weather, but can also include future predictedweather. For example, if the user is at work, and there is heavy rainforecasted for the time frame when the user typically heads home, the atleast one processor 301 can generate a candidate message like “It'ssupposed to rain really hard this afternoon, I'll probably be home alittle late” intended for the spouse or children of the user. Furtherstill, identifying the weather does not need to be limited toidentifying the weather in the immediate region of system 300, but canalso include other regions relevant to the user. For example, if theuser is at work, and there is heavy hail forecasted in the region of theuser's home, the at least one processor 301 may generate a candidatemessage like “Heavy hail forecasted for this afternoon, make sure to putyour car in the garage” directed towards the spouse or children of theuser.

Identifying the environmental context parameter may also include the atleast one processor 301 identifying the physical environment of the userand/or a current activity being performed by the user. For example,system 300 may include a physical environment sensor 306, such as acamera or microphone, in order to capture data indicative of thephysical environment of the user. Such a camera may capture visual dataindicative of the physical environment of the user. Alternatively oradditionally, a microphone of system 300 may capture audio indicative ofthe of user's physical environment. The at least one processor 301 cananalyze this data indicative of the user's physical environment todetermine the physical environment of the user, and potentially acurrent activity being performed by the user, as discussed in theseveral examples below. The camera and/or microphone could be carried,for example, on a support structure to be worn on the head of a user,such as a WHUD. Alternatively, if the system 300 does not include asupport structure to be worn on a head of a user, the camera and/ormicrophone can be carried on any appropriate component in system 300,such as on a smartphone, smartwatch, or control ring. The at least oneprocessor 301 can also be carried by the support structure, or can beincluded in a separate peripheral device, as described above regardingFIG. 1B, for example.

For example, the at least one processor 301 may determine that thephysical environment of the user is the inside of a vehicle, and thatthe user is currently driving, and generate a message like “I'm drivingnow, I'll call you later”, or “Driving home, see you soon.” The at leastone processor 301 may determine that the user is in a vehicle anddriving using many different techniques, for example: analyzing theuser's visual surroundings based on images or video captured by at leastone camera of system 300 and detecting key features of a vehicle such asa steering wheel, rear view mirrors, or side mirrors, among otherfeatures; analyzing the user's auditory surroundings based on audiocaptured by at least one microphone of system 300, such as to detect thesound of a vehicle engine, the sound of wind passing by the vehicle, orthe sound of traffic, among others; analyzing the user's speed and/oracceleration as captured by an inertial motion unit and/or GPS sensor ofsystem 300; or any combination of the above. Further, the abovetechniques are not limited to driving a motor vehicle, but can includeoperation of any number of vehicles including for example a car, van,truck, bus, tractor, scooter, moped, motorcycle, airplane, helicopter,boat, ship, vessel, bicycle, or unicycle. Exemplary techniques fordetecting when a user is operating a vehicle, and appropriate actions totake upon such detection, are described in at least U.S. ProvisionalPatent Application Ser. No. 62/750,060.

As another example of a current activity of the user which could beidentified by the at least one processor 301, the at least one processor301 could determine that the user is sitting at their desk working. Thiscould be determined for example by the at least one processor 301analyzing the user's visual surroundings based on images or videocaptured by at least one camera of system 300 and detecting key featureslike a computer monitor, and/or by the at least one processor 301determining that the user is stationary based on the output of alocation sensor of system 300. If the at least one processor 301identifies that the user is currently working at their desk, butidentifies that the time is currently past the time at which the usertypically departs work based on the output of a clock of system 300, theat least one processor 301 may generate a candidate message like “Stillat work, I'll be home late” directed towards a spouse of the user.

As another example of a current activity of the user which could beidentified by the at least one processor 301, the at least one processor301 could determine that the user is eating, such as by the at least oneprocessor 301 analyzing the user's visual surroundings based on imagesor video captured by at least one camera of system 300 and detectingfood in front of the user. The at least one processor 301 could furtheridentify what food the user is eating based on the captured images orvideo, and identify where the user is eating based on an output of alocation sensor of system 300 as discussed above. If the user is eatingspaghetti at a restaurant called “The Best Pasta”, the at least oneprocessor 301 could generate a candidate message like “The spaghetti atThe Best Pasta is delicious!”.

Further, the current activity of the user is not limited only toactivities which can be identified by using a camera or microphone toidentify a physical environment of the user, but can also include anyactivity which can be identified by the at least one processor 301 basedon data from any input sensors carried by system 300. As non-limitingexamples of input sensors, system 300 may include at least one of anelectromyography sensor, a mechanomyography sensor, an accelerometer, agyroscope, an electrocardiography sensor, a photoplethysmography sensor,a blood pressure sensor, a global positioning system sensor, a compass,an eye-tracking sensor, or any combination of these sensors together.

As a further example of a current activity of the user which could beidentified by the at least one processor 301, the at least one processor301 could determine that the user is currently exercising, based on atleast a heartrate of the user detected by an electrocardiography sensoror photoplethysmography sensor, blood pressure detected by a bloodpressure sensor, muscle activations detected by an electromyographysensor and/or mechanomyography sensor, or state of arousal detected byan eye-tracking sensor detecting pupil dilation. The at least oneprocessor 301 could then generate a message like “Working out, I'll callyou soon”, or “Pumping iron, I'm awesome!”. Further, the input sensorsdo not necessarily have to be carried by system 300 itself, but couldinstead be carried by an external sensing device. For example, system300 could communicate with a sensing device such as a wearable FitBit®monitor which measures heart rate using a photoplethysmography sensor,or any similar device, to receive input sensor data from which togenerate a message. In this example, the sensing device would detectinput sensor data, such as a heart rate, and transmit this input sensordata to system 300. The at least one processor 301 of system 300 couldthen identify a current activity being performed by the user based on atleast the input sensor data received from the sensing device, andgenerate at least one candidate message based on the current activitybeing performed by the user. However, system 300 could also beconsidered as including such a sensing device.

Identifying the current activity being performed by the user or in whichthe user is engaged may also include the at least one processor 301utilizing eye tracking to determine a gaze direction of the user, inorder to determine what the user is looking at, so that the at least oneprocessor 301 can generate a message based on what the user is lookingat. Exemplary hardware to achieve this goal could include a supportstructure, such as glasses frames or WHUD frames, to be worn on a headof a user. An eye tracking system can be carried by the supportstructure and can include, for example, an infrared light source whichemits infrared light and an infrared photodetector which receivesinfrared light reflected from an eye of the user. Exemplaryconfigurations for eye tracking can be found in for example U.S.Non-Provisional patent application Ser. No. 15/167,458, U.S.Non-Provisional patent application Ser. No. 15/827,667, and U.S.Provisional Patent Application Ser. No. 62/658,436, each of which areincorporated by reference herein in their entirety. One skilled in theart will appreciate that the eye tracking hardware need not be limitedto infrared light and infrared detectors, but can utilize anyappropriate systems or wavelengths of light. The eye tracking system maybe paired with a camera on the support structure which captures at leastone image of a view seen by a user of system 300. The at least oneprocessor 301 can analyze the output from the eye tracking system andthe image(s) captured by the camera carried by the support structure, inorder to determine what the user is looking at. The at least oneprocessor 301 may then generate at least one candidate message based onwhat the user is looking at. For example, if the at least one processordetermines that the user is looking at the cheese section of a grocerystore, the at least one processor 301 could generate a message like “Doyou want mozzarella or cheddar?” The at least one processor 301 may alsoretrieve additional information from other sources, such as for examplesearching the Internet for information regarding what the user islooking at. As an example, if the user is looking at a poster for anupcoming movie called “Awesome Movie”, the at least one processor 301could search the Internet for the release date and nearby locationswhere “Awesome Movie” is playing, and generate a message like “AwesomeMovie comes out this weekend, want to get tickets to see it downtown?”

FIGS. 5A-B and 6 show exemplary implementations of user interfacesproduced by the subject systems, devices, and methods which display orpresent image representations including messages to a user in responseto execution of the method 400 (FIG. 4 ). In response to input (e.g.,selections) received via the user interfaces, a user of system 300 canprovide input through user input mechanism 305 to perform selection,such as in act 404 of method 400. For background, in the system shown inFIGS. 5A-B and 6, the user of system 300 has a calendar which has beensynchronized with system 300, which includes an entry for a meeting withBoss Jean, Coworker Bill, and Secretary Ferdinand at 11:20. The currenttime is 11:15.

FIGS. 5A and 5B represent a sequence of displays or presentations ofuser interfaces including messages which can be shown or presented tothe user, starting from FIG. 5A and proceeding to FIG. 5B. In theseFigures, a user decides that they want to start a new communication, sothey select a “Compose message” option or user selectable icon presentedto the user (e.g., shown on the output device 304 of system 300) asillustrated in FIG. 5A. System 300 then autonomously generates at leastone (preferably multiple) candidate messages for the user to choose frombased on an environmental context parameter and presents these candidatemessages (e.g., displays the candidate messages on display 304) as shownin FIG. 5B. In the case illustrated in Figure system 300 generates twomessages related to the user's upcoming meeting, and one message relatedto lunch plans. In the implementation of FIGS. 5A, 5B, a recipient ofeach candidate message is determined autonomously by the system 300, andpaired with the respective candidate message. In this way, the user canadvantageously select both the candidate message and the recipientsimultaneously, such that the user can conveniently perform a singleselection to indicate both the message and the recipient.

A recipient for each candidate message may be determined autonomously bythe at least one processor 301 based on at least the environmentalcontext parameter and/or the content of the complete message.Determining the recipient based on environmental context parameter canuse any of the hardware, sensors, and techniques discussed aboveregarding determining the environmental context parameter. For example,the environmental context can be determined at least in part based onthe output from any one of or any combination of the following: a clock,a calendar accessible by system 300, a location sensor, a camera, amicrophone, an electromyography sensor, a mechanomyography sensor, anaccelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a globalpositioning system sensor, a compass, and an eye-tracking sensor.Further, the sensor(s) could be carried by the system 300, or could becarried by an external device communicatively coupled to system 300. Asan example, the at least one processor 301 may determine that the useris currently exercising, based on at least a heartrate of the userdetected by an electrocardiography sensor or photoplethysmographysensor, blood pressure detected by a blood pressure sensor, muscleactivations detected by an electromyography sensor and/ormechanomyography sensor, or state of arousal detected by an eye-trackingsensor detecting pupil dilation. As a result, the at least one processor301 may determine that the recipient of the complete message should bethe user's fitness trainer. Additionally or alternatively, determining arecipient for each candidate message can be based on the content of therespective candidate message. For example, if a message says “Can youplease pick up the kids at school?”, the at least one processor 301 maydetermine that the recipient of the message should be a spouse of theuser, since the content of the message relates to the user's family.Discussion below related to determination of a recipient is fullyapplicable to automatic determination of a recipient, as in FIGS. 5A-5Band 6 .

FIG. 6 shows a compact implementation in which the user can select botha message to be sent and a recipient of the message with a single input,without the need to first select a “Compose message” option. System 300generates at least one candidate message, pairs the at least onecandidate message with a recipient, and presents the pairing on a homepage of the output device 304 of system 300. The user may simply selectthe paired message and recipient, quickly and efficiently.

FIG. 7 is a flowchart which illustrates an exemplary method 700 inaccordance with the present disclosure. Although method 700 is describedwith reference to system 300 shown in FIG. 3 , method 700 is equallyapplicable to any hardware arrangement having at least one processor, anon-transitory processor-readable storage medium, a wirelesscommunications module, an output device, and a user input mechanism.Method 700 can be embodied as a set of processor-executable instructionsstored on the non-transitory processor-readable storage medium 302,which can be executed by the at least one processor 301. Method 700includes acts 701, 702, 703, 704, 705, 706, 707, and 708, though thoseof skill in the art will appreciate that in alternative implementationscertain acts may be omitted and/or additional acts may be added. Thoseof skill in the art will also appreciate that the illustrated order ofthe acts is shown for exemplary purposes only and may change inalternative implementations.

Method 700 is similar to method 400 but adds additional acts.Specifically, acts 701, 702, 703, and 704 of method 700 correspond toacts 401, 402, 403, and 404 of method 400, respectively, and thus thedescription above regarding acts 401, 402, 403, and 404 fully applies tomethod 700, and will not be repeated in the interest of brevity. Theadditional acts 705, 706 and 707 focus on determining a recipient for aselected message. Specifically, at 705, the at least one processor 301determines at least one candidate recipient for the selected messagebased on at least one of a content of the selected message and theenvironmental context parameter. At 706, the output device 304 displaysor presents the at least one candidate recipient for the selectedmessage. At 707, the at least one processor 301 receives a second inputfrom the user of system 300, the second input indicative of a selectedrecipient from the at least one candidate recipient. At 708, thewireless communications module 303 transmits the selected message fromsystem 300 to the selected recipient.

As discussed above regarding FIG. 1B, although it is possible to performall of the acts of method 700 on a single device, it is also possible toremove some of the hardware from the presentation device (such as aWHUD), and utilize a peripheral device comprising said hardware instead,in order to reduce bulk in the presentation device. In the case ofmethod 700, acts 701, 702, 704, 705, and 707 could be performed by aprocessor of a WHUD, acts 703 and 706 could be performed by an outputdevice of the WHUD, and act 708 could be performed by a wirelesscommunications module of a peripheral device communicatively coupled tothe WHUD, such that the processing and presenting acts are performed onthe WHUD, and the peripheral device is used as a wireless transmissiongateway. Alternatively, acts 701, 702, 704, 705, and 707 could beperformed by at least one processor on a peripheral device, acts 703 and706 could be performed by an output device on a WHUD communicativelycoupled to the peripheral device, and act 708 could be performed by awireless communications module on the peripheral device, such that theperipheral device performs processing and transmission, whereas the WHUDperforms presenting.

Determining at least one candidate recipient based on the environmentalcontext system 300, as in act 705, can utilize any of the data sourcesand sensors, as well as the techniques, discussed above with regards togenerating at least one candidate message based on at least theenvironmental context parameter. For example, the environmental contextcan be determined at least in part based on the output from any one ofor any combination of the following: a clock, a calendar accessible bysystem 300, a location sensor, a camera, a microphone, anelectromyography sensor, a mechanomyography sensor, an accelerometer, agyroscope, an electrocardiography sensor, a photoplethysmography sensor,a blood pressure sensor, a global positioning system sensor, a compass,and an eye-tracking sensor. Further, the sensor(s) could be carried bythe system 300, or could be carried by an external devicecommunicatively coupled to system 300.

FIGS. 8A-8C and 9A-9B show exemplary implementations of user interfacesproduced by the subject systems, devices, and methods which display orpresent image representations including messages to a user in responseto execution of the method 700 (FIG. 7 ). In response to input receivedvia the user interfaces, a user of system 300 can provide input throughuser input mechanism 305 to perform selection, such as in acts 704 and707 of method 700. For background, in the system shown in FIGS. 8A-8Band 9A-9B, the user of system 300 has a calendar which has beensynchronized with system 300, which includes an entry for a meeting withBoss Jean, Coworker Bill, and Secretary Ferdinand at 11:20. The currenttime is 11:15. In the implementation shown in FIGS. 8A-8C, a userdecides that they want to start a new communication, so they select a“Compose message” user selectable icon shown on the output device 304 ofsystem 300 in FIG. 8A. System 300 then autonomously generates at leastone (preferably multiple) candidate messages for the user to choose frombased on an environmental context parameter and displays or presents thegenerated candidate messages, e.g., displayed on display 304 as shown inFIG. 8B. In the case of FIG. 8B, system 300 generates two messagesrelated to the user's upcoming meeting, and one message related to lunchplans. The user selects a message, and system 300 then displays a listof candidate recipients for the message on display 304. The list ofrecipients could be an alphabetical list of the user's contacts, but ispreferably sorted and/or filtered based on the content of the selectedmessage or the environmental context parameter in order to only displayrelevant recipients and/or display the most relevant recipients first.In the case of FIG. 8C, since the user chose a message related to theupcoming meeting, the candidate recipients are the people who are alsoscheduled for the upcoming meeting.

FIGS. 9A-B show a compact user interface in which the user can select amessage to be sent without the need to first select a “Compose message”user selectable icon. Instead, at least one candidate message can bepresented or displayed on a main or “home” page of the output device 304of system 300 as shown in FIG. 9A, allowing the user to quickly selectthe message to send. Subsequently in FIG. 9B, at least one candidaterecipient for the selected message is presented or displayed on theoutput device 304 for the user to select.

Specific examples of how the at least one candidate recipient can bedetermined based on the environmental context data of system 300 arediscussed in more detail below, and are fully applicable to thedetermination of candidate recipients as in FIGS. 7, 8A-8C, and 9A-B.

If the acts of method 700 are performed in the order shown in FIG. 7 ,at 705, the at least one candidate recipient can be determined based onnot only the environmental context parameter, but also on the content ofthe selected message, since the at least one recipient is determinedafter the message is selected. For example, with reference to FIG. 8B,if the user selects the “See you in 5 minutes” message, referring to ameeting scheduled in a calendar accessible by system 300, the at leastone processor 301 of system 300 could determine that since the contentof the message is related to the meeting, the most relevant recipientswill be other people scheduled to attend the meeting. Thus, the at leastone processor 301 accesses the calendar (i.e., data indicatingenvironmental context parameter), determines who else is scheduled toattend the meeting, and displays a list of user selectable iconscorresponding to respective candidate recipients which includes theother attendees of the meeting. As another example, if the user selectsa message like “Driving now, see you soon”, the at least one processor301 can analyze which person or persons the user is heading towards, inorder to determine at least one appropriate recipient or candidaterecipients. For example, the at least one processor 301 could receivethe outputs of at least a location sensor (e.g., GPS receiver),accelerometer, and a compass, to determine that the user is heading awayfrom their workplace towards their home, and decide that residents ofthe user's home such as a spouse or children should be candidaterecipients of the message.

One skilled in the art will appreciate that in the context of method700, certain acts may be omitted and/or additional acts may be added.Those of skill in the art will also appreciate that the illustratedorder of the acts is shown for exemplary purposes only and may bechanged.

In one such reordered method, method 700 can be reordered such that theacts are performed in the order: 701, 705, 706, 707, 702, 703, 704, 708.This order of acts could correspond to the sequence of user interfacesshown in FIGS. 10A-10C, in which at least one candidate recipient willbe determined, and a recipient selected by the user, before generationof at least one candidate message. In response to the user interfaces, auser of system 300 can provide input through user input mechanism 305 toperform selection, such as in acts 704 and 707 of method 700. Forbackground, in the implementation illustrated in FIGS. 10A-10C, the userof system 300 has a calendar which has been synchronized with system300, which includes an entry for a meeting with Boss Jean, CoworkerBill, and Secretary Ferdinand at 11:20. The current time is 11:15.Because the at least one candidate recipient must be determined beforethe at least one candidate message is generated, the at least oneprocessor 301 will not be able to use content of the candidate messageto determine at least one candidate recipient. The list of candidaterecipients could be a list of all of the user's contacts, but ispreferably filtered and/or sorted to display the most relevant candidaterecipients first based on at least an environmental context parameter.In FIG. 10B, for example, since the user has a meeting scheduled in 5minutes, other scheduled attendees of the meeting are displayed.Further, the user's husband Bert could be displayed, either becausesystem 300 assigns a high priority to close contacts like spouses,and/or because the user tends to message their husband Bert frequentlyor recently. Once the user selects their husband Bert as the recipientof the new message, system 300 autonomously generates at least one,preferably multiple, candidate messages and presents these candidatemessages on display 304 for the user to choose from. Since husband Bertwas selected as the recipient of the new message, system 300 willgenerate messages that are pertinent to husband Bert, such as a lunchinvitation, rather than irrelevant messages such as those that would berelated to the user's upcoming meeting.

As further examples, the at least one candidate recipient can bedetermined based on the environmental context parameter. For example, ifthe at least one processor 301 determines that the user is currentlyexercising based on at least one of a heartrate of the user detected byan electrocardiography sensor or photoplethysmography sensor, bloodpressure detected by a blood pressure sensor, muscle activationsdetected by an electromyography sensor and/or mechanomyography sensor,or state of arousal detected by an eye-tracking sensor detecting pupildilation, the at least one processor 301 may determine that only peoplewith non-professional relationships to the user may be candidaterecipients for a message. For example, the at least one processor 301may determine that only personal friends of the user, not workcolleagues, are candidate recipients for a message. Further, the atleast one processor 301 can make more accurate determinations ofcandidate recipients based on the amount of data available regarding theuser's contacts. For example, if the user has a contact like “PersonalTrainer Herbert”, the at least one processor 301 could determine thatPersonal Trainer Herbert is very likely an appropriate recipient of amessage when the user is exercising.

As an additional alternative, acts 703 and 706 may be performedsubstantially simultaneously or concurrently, and acts 704 and 707 maybe performed substantially simultaneously or concurrently. Thisalternative implementation could correspond to the implementation shownin FIGS. 5A-5B or the implementation shown in FIG. 6 , in which acandidate message is generated and a candidate recipient is determined,for simultaneous presentation or display to the user. The user can thenselect the paired message and recipient with a single input. One skilledin the art will appreciate that acts 702 and 705 do not necessarily haveto be simultaneous or concurrent. Rather, the at least one processor 301could generate the candidate messages first, and determine a recipientfor each of the candidate messages based on at least one of a content ofthe candidate message and the environmental context parameter, then theoutput device 304 could present or display both the candidate messagesand the paired recipients simultaneously or concurrently. The at leastone processor 301 could then receive an input from the usersimultaneously selecting both a message and recipient. Similarly, the atleast one processor 301 could determine at least one candidate recipientfirst, then generate candidate messages for each candidate recipientbased on at least one of the environmental context parameter and therelationship between the user and respective candidate recipient, thenthe output device 304 could present both the candidate recipients andthe paired candidate messages simultaneously or concurrently. The atleast one processor 301 could receive an input from the usersimultaneously selecting both a recipient and a message.

In addition to the many possibilities which can be included inidentifying the environmental context data of system 300, identifyingthe environmental context parameter may also include the at least oneprocessor 301 analyzing a messaging history of system 300. For example,in the implementations where the message is generated before therecipient is determined, such as those shown in FIGS. 8A-8C and 9A-9B,after the user selects the message and the recipient, the at least oneprocessor 301 may analyze the communication history between the user andthe selected recipient. In particular, the at least one processor 301may analyze the messages sent between the user and the recipient, todetermine a communication vernacular typically used between the user andrecipient. For example, if the user and recipient are close friends, themessage history will likely include messages with a casual conversationtone. On the other hand, if the recipient is the user's employer, themessage history will likely include messages with a professional tone.After analyzing the communication vernacular between the user and therecipient, the at least one processor 301 may autonomously modify theselected message based on the communication vernacular. For example, ifthe user selected a message which uses formal language like “Would youlike to eat lunch together today?”, and selected a recipient of “Bestfriend Willard”, the at least one processor 301 could review the messagehistory between the user and Willard, and discover that the user andWillard frequently communicate with casual slang or colloquially. The atleast one processor 301 could then autonomously modify the message touse casual language like “Wanna grab some grub bro?”.

Alternatively, in the implementations where the recipient is determinedbefore the message is generated, such as those of FIGS. 10A-10C, the atleast one processor 301 can analyze the message history between the userand the selected recipient to determine an appropriate communicationvernacular, and can utilize the determined vernacular when generatingthe at least one candidate message.

Analyzing the message history between the user and a candidate or anintended recipient of a message as discussed above can include the atleast one processor 301 analyzing a plurality or all of the messagespreviously sent between the user and the candidate or intendedrecipient, including messages sent from the user to the candidate orintended recipient, as well as messages sent from the candidate orintended recipient to the user. However, the at least one processor 301can also determine that the message most recently received by the usershould be ignored. For example, current messaging systems sometimesincorporate a “smart reply” feature, in which when a device receives amessage, the device attempts to interpret the message and prepare a fewreplies which the user may want to use as a response. However, thesesystems are limited only to analyzing a single message, and also do notadjust the prepared replies based on how long ago the message wasreceived. In the present systems, devices, and methods, the at least oneprocessor 301 can selectively choose to disregard the most recentlyreceived message or plurality of messages when generating messages, forexample by determining that the most recently received message ormessages are too old to be relevant. This decision can be based on atleast the content of the message or messages, and/or the environmentalcontext parameter. For example, if a user receives a message like “Havea nice day!” at 8:30 AM from their spouse, a typical smart reply systemwould generate replies like “You too!” or “Thanks!”, even if the userdoes not decide to send a message until 5:00 PM that night when they areheading home. At 5:00 PM, there is little meaning to responding to themessage that was received at 8:30 AM, and the user will likely want tosend a message related to a new topic, like “I'm heading home now, seeyou in 45 minutes.” The at least one processor 301 in the presentdisclosure can achieve this, by identifying the amount of time that haslapsed since the last message was received based on an output of a clockof system 300. As an example, the at least one processor 301 coulddetermine that a smart reply should not be generated for any messagethat was received more than four hours ago, and the at least oneprocessor 301 should instead focus on generating new messages.

One skilled in the art would understand that four hours is merely anexemplary time frame, and that this time frame could be adjusted to anyappropriate amount of time as determined by one skilled in the art for aspecific application. Alternatively, the present systems, devices, andmethods can also include a mechanism for evaluating a relevancepersistency of the most recently received message, wherein the at leastone processor 301 identifies at least a content of the most recentlyreceived message, and adjusts the time frame for which a smart replywill be generated based on at least the content of the most recentlyreceived message. As an example, if the at least one processor 301identifies the content of the most recently received message as justbeing a simple greeting, the at least one processor 301 may generatesmart replies to the greeting until two hours have elapsed since thegreeting was received, after which the at least one processor 301 willdisregard the greeting and generate new messages. As another example, ifthe at least one processor 301 identifies the content of the mostrecently received message as being a request to set up an appointment,the at least one processor 301 may generate smart replies indefinitelyuntil the user has responded to the appointment request, or until thetime and date for the requested appointment have passed.

Identifying the environmental context parameter may also include the atleast one processor 301 identifying a relationship between the user ofsystem 300 and the recipient of the message. For example, the at leastone processor 301 may determine that the user has a casual orprofessional relationship with the recipient based on the determinedcommunication vernacular as discussed above. Alternatively, the at leastone processor 301 may determine the relationship between the user andthe recipient based on a flag identifying the relationship. The flagcould be a user assigned flag, such as by the user manually inputtingthe relationship to the recipient when saving the recipient's contactinformation. Alternatively, the flag could have been previously set bythe at least one processor 301 when the user was sending a differentmessage to the recipient, thus saving processor power by avoidingredundant analysis.

In the implementations where the message is generated before therecipient is determined, such as those shown in FIGS. 8A-8C and 9A-9Bafter the user selects the message and the recipient, the at least oneprocessor 301 may autonomously modify the selected message based on therelationship. For example, if the user selected a message like “Wouldyou like to go out for dinner?”, and then selected a recipient of“Husband Bert”, the at least one processor 301 may autonomously modifythe selected message to instead say something more intimate and flirty,like “What's cooking, good looking?”.

In the implementations where the recipient is determined before themessage is generated, such as those of FIGS. 10A-10C, the at least oneprocessor 301 can generate the at least one messages based on therelationship between the user and the recipient.

If the at least one processor 301 is unable to confidently identify acasual vernacular or a casual relationship between the user and therecipient, the at least one processor 301 can by default generate thecandidate message based on a polite, non-casual vernacular.

To improve accuracy and functionality of the present systems, methods,and devices, machine learning can be employed to provide more dataregarding the environmental context parameter. For example, if the userfrequently messages their spouse around 5:00 PM saying “Heading home”,the at least one processor 301 may analyze and learn this behavior, andgenerate “Heading Home” as a candidate message with the user's spouse asa recipient if the clock output is around 5:00 PM, such as for examplebetween 4:00 PM and 6:00 PM Monday to Friday. As another example, the atleast one processor 301 may learn over time what recipients the usermessages the most and assign higher priority to these recipients.Further, the at least one processor 301 may analyze messages sent fromdifferent users to different recipients in order to build a morecomprehensive database on which to base the generation of messages.Specifically, the present systems and methods could be implemented in alarge range of systems, such that at least one processor of each systemanalyzes messages being sent from the systems to determine at leastcontent, tone, format, and language use when sending messages from auser to a given recipient. The at least one processor of each system mayanalyze the determined content, tone, format, and language use in thecontext of the relationship between the user and the sender in order tobuild a dataset of machine knowledge on which to base the generation ofmessages. Each system on which the present systems and methods areimplemented could have access this dataset, both to retrieve data onwhich to base messages, as well as to provide data to the dataset toimprove the dataset. In this sense, many systems can act together andpool knowledge to enhance not only their own performance, but also theperformance of every other system. Further, each system may also analyzeexternal resources to improve message generation capability. Forexample, each system may access the Internet, and analyze at least forumposts, blog Websites, comments, dictionaries, and any other appropriateonline resource in order to build a machine knowledge database on whichto base the generation of messages.

Further, machine learning can be utilized to populate information aboutthe user's habits based on the output of a location sensor of system300. For example, although the systems, methods, and devices of thesubject application allow for the user of system 300 to manually inputimportant or relevant locations, such as a home or workplace, thisinformation can also be determined autonomously. As an example, if thelocation sensor indicates that a user is at a first locationcorresponding to a first set of GPS coordinates 5 times per week, and isat a second location corresponding to a second set of GPS coordinatesevery day of the week, the at least one processor 301 may determine thatthe first set of GPS coordinates correspond to the user's workplace, andthe second set of GPS coordinates correspond to the user's home.Further, this determination does not necessarily have to be made by theat least one processor 301, but instead can be made by a remote serveror third party service, such as Pilgrim by Foursquare®.

Additionally, the at least one processor 301 of system 300 mayincorporate deliberate variability in the generation of the messages.For example, if a user sends their spouse a cute or romantic messageevery day at 3:00, the at least one processor 301 may learn to generatedifferent cute or romantic messages, to ensure that the spouse receivesinteresting variations of messages daily, thus preventing spouseboredom, and saving the marriage of the user and spouse. The at leastone processor 301 may use machine learning to enable this variability.For example, the at least one processor 301 may analyze the user'sprevious messages to determine tone and recipient, analyze messages sentby other systems with similar tone and recipient, and build a library ofmessage ideas and formats on which to base the messages. The at leastone processor 301 may also connect to the Internet to analyze externalresources to determine appropriate conversation tones, formats, andlanguage use on which to base the messages.

When generating at least one candidate message or determining at leastone candidate recipient, the environmental context can be anycombination of the different examples discussed above, in addition toany other appropriate environmental context known to system 300. Thesecombinations can be achieved by using weighted confidence scores. Forexample, if the location sensor alone is used to determine that the useris at work, it would be difficult for the at least one processor 301 totell whether the user is scheduled for a meeting, or is heading home.However, by combining multiple types of environmental context data, theat least one processor 301 can more confidently determine the status ofthe user, and can thus generate more accurate candidate messages. Inreference to the above example, if the location sensor indicates thatthe user is currently at their workplace, and this is the onlyenvironmental context data available to the at least one processor 301,then the at least one processor 301 will assign a low confidence scoreto the possibility that the user is about to leave work. However, if theat least one processor 301 also has access to data output from a clockof system 300 which says the current time is 5:00 PM, the at least oneprocessor 301 can assign a higher confidence score to the possibilitythat the user is about to leave work. Further still, if the at least oneprocessor 301 has access to a calendar of the user, and there are nomeetings scheduled for the rest of the day, the at least one processor301 can assign an even higher confidence score to the possibility thatthe user is about to leave work. On the other hand, if the at least oneprocessor 301 has access to data captured by a camera on system 300which shows that the user is currently in the middle of an impromptumeeting, the at least one processor 301 can assign a lower confidencescore to the possibility that the user is about to leave work. The atleast one processor 301 may use this strategy to analyze many differentsituations and possibilities, and then generate at least one candidatemessage based on the possibilities with the highest confidence scores.

If the at least one processor 301 is unable to assign a high confidencescore to a possible situation or candidate message, the at least oneprocessor 301 can resort to generating more generic candidate messages,such as generic greetings based on the current time of day or currentdate, e.g. “Good Morning” or “Happy Monday!”.

Confidence scores can similarly be used when determining at least onecandidate recipient for the message. For example, when a location sensorof system 300 indicates to the at least one processor 301 that the useris currently at a school attended by the user's children, the at leastone processor 301 will assign a low confidence score to the possibilitythat the user's boss should be the recipient of the message, and willassign a high confidence score to the possibility that the user's spouseshould be a recipient of the message. Further, if the content of themessage relates to the user's children, the at least one processor 301can assign an even lower confidence score to the possibility that theuser's boss should be a recipient of the message, and will assign aneven higher confidence score to the possibility that the user's spousewill be a recipient of the message.

If the at least one processor 301 is unable to assign a high confidencescore to a possible recipient for the message, the at least oneprocessor 301 can cause the output device 304 to display an alphabeticallist or grouping of the user's contacts. Alternatively, the at least oneprocessor 301 can list the contacts starting with the contacts with thehighest confidence scores.

In addition or alternatively to confidence scores, the at least oneprocessor 301 can generate the at least one candidate message usingweighted priority scores. For example, if the user has a meetingscheduled in one hour, the at least one processor 301 may assign a highpriority score to generation of messages relating to this meeting, and alow priority score to generation of messages relating to other topics.Priority can be determined by, for example, by considering topicsrelated to the user's professional and work life to be more importantthan topics related to the user's personal life, or vice-versa.

Priority scores can also be used by the at least one processor 301 whendetermining candidate recipients for a message. For example, if alocation sensor of system 300 outputs that the user is currently at ahospital, the at least one processor 301 can assign a very high priorityscore to the user's spouse or other family members as being candidaterecipients for a message.

Both confidence scores and priority scores can be influenced by a user'shabits. For example, if the at least one processor 301 assigns a highconfidence score to a user's spouse being a candidate recipient for amessage as the user leaves work, but the user ignores this option andinstead selects their friend as the recipient for a message after work,the at least one processor 301 can learn and assign a higher confidencescore to the friend being a candidate recipient for a message afterwork, while assigning a lower confidence score to the user's spousebeing a candidate recipient of a message when the user leaves work. Asanother example, consider a case where the at least one processor 301assigns a high priority score to messages based on the user'sprofessional life and a low priority score to messages related to theuser's personal life, but the user frequently does not select candidatemessages based on their professional life, and instead consistentlyselects candidate messages related to their personal life. In thisexample, the at least one processor 301 can learn and adapt to theuser's habits, and assign a higher priority score to messages pertainingto the user's personal life, while assigning a low priority score tomessages pertaining to the user's professional life.

Weighting of the confidence scores and priority scores can involvebinary decision making. For example, the at least one processor 301 candetermine that if the time is anywhere between 11:00 AM and 12:00 PM,the confidence score for generating messages related to lunch should beincreased by a set amount. Conversely, the at least one processor candetermine that if the time is anywhere outside of 11:00 AM and 12:00 PM,the confidence score for generating messages related to lunch should notbe increased, or even should be decreased by a set amount. However, thiscreates a situation where at 10:59 AM, the confidence score forgenerating messages related to lunch will be low, even though the timeis still close enough to lunch that the user might be consideringsending lunch related messages. To improve accuracy of the confidencescores and priority scores, the scores can be generated using non-binaryrange-based weighting. Specifically, instead of increasing or decreasingthe confidence or priority score by a set amount, the confidence orpriority score can be adjusted based on how far the environmentalcontext parameter of interest is from a “peak” value. In the aboveexample, “peak” time for lunch related messages could be set at 11:55AM. In this case the at least one processor 301 will increase theconfidence score for generating messages related to lunch based on howclose the time is to 11:55 AM. For example, the confidence score forgenerating messages related to lunch could be low at 10:00 AM, moderateat 11:00 AM, high at 11:30 AM, peak at 11:55 AM, high at 12:15 PM,moderate at 12:30 PM, and low at 1:00 PM. Additionally, the confidenceor priority score can also be adjusted based on a “start” value belowthe “peak” value and an “end” value above the “peak” value. Theconfidence score or priority score can be decreased or held constant ifthe environmental context parameter is outside the range encompassed bythe start value and the end value. In the above example, the start valuecould be set at 10:00 AM, the end value could be set at 1:00 PM, and thepeak value could be set at 11:55 AM. The confidence score for generatinga message related to lunch could be set to 0 (i.e., no messages relatedto lunch will be generated) if the time is outside the range of 10:00 AMto 1:00 PM, and could be increased if the time is within the range of10:00 AM to 1:00 PM, reaching the highest value at the peak time of11:55 AM. For example, the confidence score for generating messagesrelated to lunch could be Oat 9:30 AM, low at 10:00 AM, moderate at11:00 AM, high at 11:30 AM, peak at 11:55 AM, high at 12:15 PM, moderateat 12:30 PM, low at 1:00 PM, and Oat 1:30 PM. One skilled in the artwill readily appreciate that all of the example times listed herein aremerely exemplary, and any appropriate times could be chosen asappropriate for a specific application or for a specific user.Preferably, the at least one processor 301 can autonomously learn auser's habits and schedule over time, and adjust the start time, peaktime, and end time of any given message type accordingly.

In some implementations, generation of messages can be based on at leastone Bayesian inference model. In one such implementation, a library ofphrases could be stored, along with at least one relevance score. Eachphrase could have an accompanying time-relevance score, which models therelevance of the phrase based on the time of day. For example, a phrase“Heading home” could have a relevance score modelled by a Gaussian curvewith a peak at 5:30 PM. Further, each phrase could also have acategory-relevance score, which models the relevance of the phrase basedon a category from the library of categories. For example, the phrase“Heading home” could have a category-relevance score which is high for a“commute” category, but low, zero, or no category-relevance score for a“meetings” category. Further, each phrase could also have arecipient-relevance score, which models the relevance of the phrasebased on an intended recipient. For example, the phrase “Heading home”could have a recipient relevance score which is high for members of auser's family, but low for everyone else.

A total-relevance score can be calculated from the different relevancescores. In the above example for the phrase “Heading home”, thetime-relevance score, the category-relevance score, and therecipient-relevance score can be multiplied together to obtain atotal-relevance score. For example, if it is 5:30 PM, for a messagecategory of “commute”, with an intended recipient of “Spouse”, theresulting total-relevance score can be high, because each of theindividual relevance scores are high.

A given implementation can utilize as many or as few different relevancescores as is desired or appropriate for a given application. Further,even if many different relevance scores are stored for each phrase, eachrelevance score does not necessarily have to be used to determine atotal relevance score for the phrase. For example, if message content isselected before a recipient is determined, a recipient-relevance scorecan be left out of the calculation of the total relevance score.

FIG. 11 is a flowchart which illustrates another exemplary method 1100in accordance with the present systems, devices, and methods. The method1100 utilizes a “fill in the blank” style of message generation.Although method 1100 is described with reference to system 300 shown inFIG. 3 , method 1100 is equally applicable to any hardware arrangementhaving at least one processor, a non-transitory processor-readablestorage medium, a wireless communications module, an output device, anda user input mechanism. Method 1100 can be embodied as a set ofprocessor-executable instructions stored on the non-transitoryprocessor-readable storage medium 302, which can be executed by the atleast one processor 301. Method 1100 includes acts 1101, 1102, 1103,1104, 1105, 1106, and 1107, though those of skill in the art willappreciate that in alternative implementations certain acts may beomitted and/or additional acts may be added. Those of skill in the artwill also appreciate that the illustrated order of the acts is shown forexemplary purposes only and may change in alternative implementations.

At 1101, at least one processor 301 generates a first portion of amessage. Generation of the first portion of the message can be achievedby the at least one processor 301 identifying the environmental contextparameter, and generating the first portion of the message based on theenvironmental context parameter, using any of the means discussed indetail above. For example, the environmental context can be determinedat least in part based on the output from any one of or any combinationof the following: a clock, entries in a calendar accessible by system300, a location sensor, a camera, a microphone, an electromyographysensor, a mechanomyography sensor, an accelerometer, a gyroscope, anelectrocardiography sensor, a photoplethysmography sensor, a bloodpressure sensor, a global positioning system sensor, a compass, and aneye-tracking sensor. One exemplary generation of a first portion of amessage could be if the location sensor of system 300 outputs that theuser is near a restaurant called “Yum Food”, and the time is 11:00 PM,the at least one processor 301 could generate a first portion of amessage that says “Let's eat at Yum Food”

At 1102, an output device 304 displays or presents the first portion ofthe message. At 1103, the at least one processor 301 generates at leasttwo candidate second portions to complete the message based on at leastthe first portion of the message, and optionally the environmentalcontext parameter. In the above example where the first portion of themessage is “Let's eat at Yum Food_”, the at least one processor 301 maygenerate candidate second portions to complete the message like“tomorrow”, “this weekend”, or “sometime”, for example. Generation ofthe at least two candidate second portions to complete the message canbe achieved by the at least one processor 301 identifying theenvironmental context parameter, and generating the at least twocandidate second portions to complete the message based on theenvironmental context parameter, using any of the criteria or algorithmsdiscussed in detail above. For example, if the first portion of themessage is “I'm feeling_”, and if the at least one processor 301determines that over 50% of the user's calendar for the day is full, theat least one processor 301 may generate “super busy” as one of the atleast two candidate second portions to complete the message. One skilledin the art will appreciate that indicating that the user is super busyif 50% of the user's calendar is full is merely an exemplary number, andin practice this percentage could be adjusted to any amount based on theuser's typical schedule and habits.

At 1104, the output device 304 displays or presents the at least twocandidate second portions. The at least two candidate second portionscan be text-based or be images. At 1105, the at least one processor 301receives a first input from a user of system 300, the first inputindicative of a selected portion from the at least two candidate secondportions. At 1106, the at least one processor 301 generates a completemessage by combining the first portion and the selected second portion.In the above example where the first portion of the message is “Let'seat at Yum Food”, and the candidate second portions to complete themessage are “tomorrow”, “this weekend”, or “sometime”, if the userprovides an input indicative of the second portion “this weekend”, theat least one processor 301 can combine “Let's eat at Yum Food_” and“this weekend” to generate a complete message which says “Let's eat atYum Food this weekend”. At 1107, a wireless communications module 303transmits the complete message from system 300 to one or a plurality ofother systems or devices.

A candidate recipient for the complete message may be determinedautonomously by the at least one processor 301 based on at least one ofthe environmental context parameter or the content of the completemessage, using any of the hardware, sensors, and techniques discussedabove regarding determining recipients for a message. For example, theenvironmental context can be determined at least in part based on theoutput from any one of or any combination of the following: a clock, acalendar accessible by system 300, a location sensor, a camera, amicrophone, an electromyography sensor, a mechanomyography sensor, anaccelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a globalpositioning system sensor, a compass, and an eye-tracking sensor.Further, the sensor(s) could be carried by the system 300, or could becarried by an external device communicatively coupled to system 300. Asan example, the at least one processor 301 may determine that the useris currently exercising, based on at least a heartrate of the userdetected by an electrocardiography sensor or photoplethysmographysensor, blood pressure detected by a blood pressure sensor, muscleactivations detected by an electromyography sensor and/ormechanomyography sensor, or state of arousal detected by an eye-trackingsensor detecting pupil dilation. Additionally, if the content of thecomplete message is “I'm feeling pumped!”, the at least one processor301 may determine that the recipient of the complete message should bethe user's fitness trainer. As another example, if the complete messagesays “Can you please pick up the kids at school?”, the at least oneprocessor 301 may determine that the recipient of the message should bea spouse of the user, since the complete message pertains to the familyof the user.

As discussed above regarding FIG. 1B, although it is possible to performall of the acts of method 1100 on a single device, it is also possibleto remove some of the hardware from a presentation device (such as aWHUD), and utilize a peripheral device comprising said hardware instead,in order to reduce bulk in the presentation device. In the case ofmethod 1100, acts 1101, 1103, 1105, and 1106 could be performed by aprocessor of a WHUD, acts 1102 and 1104 could be performed by an outputdevice of the WHUD, and act 1107 could be performed by a wirelesscommunications module of a peripheral device communicatively coupled tothe WHUD, such that the processing and presenting acts are performed onthe WHUD, and the peripheral device is used as a wireless transmissiongateway. Alternatively, acts 1101, 1103, 1105, and 1106 could beperformed by at least one processor on a peripheral device, acts 1102and 1104 could be performed by an output device on a WHUDcommunicatively coupled to the peripheral device, and act 1107 could beperformed by a wireless communications module on the peripheral device,such that the peripheral device performs processing and transmission,whereas the WHUD performs presenting.

FIGS. 12A-12C show exemplary implementations of user interfaces producedby the system 300 which employ a “fill in the blank” structure tocompose a message, which is displayed or presented to a user of system300. In response input (e.g., selections) made via the user interfaces,a user of system 300 can provide input through user input mechanism 305to perform selection, such as in act 1105 of method 1100. In FIG. 12A, auser decides they want to send a message, and so selects the “ComposeMessage” user selectable icon presented (e.g., displayed on display 304of system 300) or projected to a retina of the user. FIG. 12B shows apresentation or display of a number of user selectable messages,including an “I'm Feeling_” option. When the user selects the “I'mfeeling_” candidate message, an additional user interface 12C ispresented in which the user can select an option to fill in the blank inthe selected candidate message. In the case of FIG. 12C, the optionsinclude a variety of different feelings which may be appropriate for thesituation. This implementation is not limited to feelings, but can beused to provide many options for the user to fill in the blank. Forexample, the user could select a message option of “Heading home, behome by_”, and the output device 304 shown in FIG. 12C would presentseveral options for times instead of options for feelings. As anotherexample, the user could select a message icon that states: “Can youplease pick up the kids at_”, and the output device 304 shown in FIG.12C would display options for places where the kids might be instead offeelings.

FIG. 13 is a flowchart which illustrates an exemplary method 1300 inaccordance with the present disclosure. Although method 1300 isdescribed with reference to the system 300 shown in FIG. 3 , method 1300is equally applicable to any hardware arrangement having at least oneprocessor, a non-transitory processor-readable storage medium, awireless communications module, an output device, and a user inputmechanism. Method 1300 can be embodied as a set of processor-executableinstructions stored on the non-transitory processor-readable storagemedium 302, which can be executed by the at least one processor 301.Method 1300 includes acts 1301, 1302, 1303, 1304, 1305, 1306, 1307,1308, 1309, and 1310 though those of skill in the art will appreciatethat in alternative implementations certain acts may be omitted and/oradditional acts may be added. Those of skill in the art will alsoappreciate that the illustrated order of the acts is shown for exemplarypurposes only and may change in alternative implementations.

Method 1300 is similar to method 1100 but adds additional acts.Specifically, acts 1301, 1302, 1303, 1304, 1305, and 1306 of method 1300correspond to acts 1101, 1102, 1103, 1104, 1105, and 1106 of method1100, respectively, and thus the disclosure above regarding acts 1101,1102, 1103, 1104, 1105, and 1106 fully applies to method 1300 and thediscussion will not be repeated here in the interest of brevity. Theadditional acts 1307, 1308, 1309, and 1310 focus on determining arecipient for the complete message and sending the complete message tothe determined or intended recipient. Specifically, at 1307, the atleast one processor 301 determines at least one candidate recipient forthe complete message based on at least one of a content of the completemessage and the environmental context parameter. At 1308, the outputdevice presents or displays the at least one candidate recipient for thecomplete message. At 1309, the at least one processor 301 receives asecond input from the user of the system 300, the second inputindicative of a selected recipient from the at least one candidaterecipient. At 1310, the wireless communications module transmits thecomplete message from system 300 to the selected recipient.

These additional acts 1307, 1308, 1309, and 1310 are very similar toacts 705, 706, 707, and 708 in method 700, and thus acts 1307, 1308,1309, and 1310 can utilize any of the data sources and sensors, as wellas the techniques, discussed above with regards to determining at leastone candidate recipient based on at least the environmental contextparameter and/or the content of the message in the implementation shownin FIG. 7 . For example, the environmental context can be determined atleast in part based on the output from any one of or any combination ofthe following: a clock, a calendar accessible by system 300, a locationsensor, a camera, a microphone, an electromyography sensor, amechanomyography sensor, an accelerometer, a gyroscope, anelectrocardiography sensor, a photoplethysmography sensor, a bloodpressure sensor, a location sensor, a global positioning system sensor,a compass, and an eye-tracking sensor. Further, the sensor(s) could becarried in system 300, or could be carried by a peripheral devicecommunicatively coupled to system 300.

As discussed above regarding FIG. 1B, although it is possible to performall of the acts of method 1300 on a single device, it is also possibleto remove some of the hardware from a presentation device (such as aWHUD), and utilize a peripheral device comprising said hardware instead,in order to reduce bulk in the presentation device. In the case ofmethod 1300, acts 1301, 1303, 1305, 1306, 1307, and 1309 could beperformed by a processor of a WHUD, acts 1302, 1304, and 1308 could beperformed by an output device of the WHUD, and act 1310 could beperformed by a wireless communications module of a peripheral devicecommunicatively coupled to the WHUD, such that the processing andpresenting acts are performed on the WHUD, and the peripheral device isused as a wireless transmission gateway. Alternatively, acts 1301, 1303,1305, 1306, 1307, and 1309 could be performed by at least one processoron a peripheral device, acts 1302, 1304, and 1308 could be performed byan output device on a WHUD communicatively coupled to the peripheraldevice, and act 1310 could be performed by a wireless communicationsmodule on the peripheral device, such that the peripheral deviceperforms processing and transmission, whereas the WHUD performspresenting.

FIGS. 14A-14D show exemplary implementations of user interfaces producedby the system 300 which employ a “fill in the blank” structure tocompose a message, which is displayed or presented to a user of system300. In response to input (e.g., selections) provided via the userinterfaces, a user of system 300 can provide input through user inputmechanism 305 to perform selection, such as in acts 1305 and 1309 ofmethod 1300. The user interfaces of FIGS. 14A-14D are, similar in somerespects to those to FIGS. 12A-12C, but include an additional interfacewhich allows the user to select an intended recipient of the message. InFIG. 14A, a user interface allows a user to select or indicate that theywant to send a message by selecting the “Compose Message” userselectable icon presented or displayed on display 304 of system 300.FIG. 14B shows a user interface which presents or displays a number ofmessages, including, e.g., “I'm Feeling_” user selectable message. Whenthe user selects the “I'm feeling_” candidate message, an additionaluser interface 14C is presented which allows the user to select anoption to fill in the blank in the (previously) selected message. In thecase of FIG. 14C, the user selectable icons correspond to a variety ofdifferent feelings which may be appropriate for the situation. Thisimplementation is not limited to feelings, but can be used to providemany options for the user to fill in the blank. For example, the usercould select a message option of “Heading home, be home by_”, and theuser interface (FIG. 14C) would display several user selectable iconscorresponding to respective times (e.g., 5:30 PM; 6 PM; 6:30 PM, PM)instead of options for feelings. As another example, the user couldselect a message icon that corresponds to the message: “Can you pleasepick up the kids at_”, and the subsequent user interface (FIG. 14C) maypresent or display a number of user selectable icons that correspond tovarious different places where the kids might be located. In FIG. 14D,the user interface 304 presents or displays a list of candidate intendedrecipients for the complete message, from which the user can choose.

FIGS. 15A-15D show exemplary implementations of user interfaces producedby system 300 which employ a “fill in the blank” structure to compose amessage, which is displayed or presented to a user of system 300. Theuser interfaces of FIGS. 15A-15D are similar in some respects to thoseof FIGS. 12 and 14 , and further allow the user to select images for thesecond portion of the message. Particularly, the user interface of FIG.14C is replaced with a user interface illustrated in FIG. 15C whichdisplays images for filling in the blank instead of text. The userinterface of FIG. 15C could display emoji's, emoticons, drawings,computer-generated renderings, photographs, or any other appropriateform of image, e.g., an image without any characters or text; or animage that includes or is combined with characters. This is particularlyeffective when a user selects a message icon that corresponds to amessage such as: “I'm feeling_”, but could also be implemented for anyother appropriate message type. For example, in the example where theuser selects a message icon that corresponds to a message: “Can youplease pick up the kids at_”, the user interface illustrated in FIG. 15Ccould display images of places where the kids could be, such as a piano,a swimming pool, a soccer ball, a school, or any other appropriateimage. Again, the selection of the provided choices of possiblelocations can be informed by, for instance appointments in a calendar, aset of stored contact information, a weekly scheduled discerned fromprevious messages, recent messages, current location(s), current date,current day of the week, current time of day, current season, holidays,special events in the geographic area, etc.

The present systems, devices, and methods also include an “explore”mode, for generating activity suggestions based on the user's location.As an illustrative example, in the implementations of FIGS. 11 and 13 ,the at least one processor 301 could generate a first portion of amessage like “Let's meet at_”. As candidate second portions to completethe message, the at least one processor 301 could generate portionsbased on the names of venues within the region of the user. For example,the at least one processor 301 could generate candidate second portionsbased on the names of restaurants within a 500 meter radius of the user.As an example, if the user is within 500 meters of restaurants called“Yummy Pasta”, “Delicious Tacos”, and “Amazing Barbeque”, as indicatedby a location sensor in system 300, the at least one processor 301 couldgenerate candidate second portions of “Yummy Pasta”, “Delicious Tacos”,and “Amazing Barbeque”. If the user selects the second portion“Delicious Tacos”, the at least one processor 301 can generate thecomplete message of “Let's meet at Delicious Tacos” by combining thefirst and second portions. This implementation advantageously can exposethe user to new and interesting options that they may not have beenaware of. Further, a similar “explore” mode could be implemented in themethods illustrated in FIGS. 4 and 7 , except instead of generating afirst portion of a message like “Let's meet at_”, and requiring a userto select a second portion to complete the message, the at least oneprocessor 301 could generate a message like “Let's get dinner at AmazingBarbeque tonight” from the outset, if the user is near a restaurantcalled “Amazing Barbeque”.

One skilled in the art will appreciate that the above implementationsare not limited to restaurants, but could include any other venues andlocations such as art galleries, theatres, nature parks, museums,concerts, festivals, or any other type of venue or event. Additionally,one skilled in the art will appreciate that the radius of 500 meters isexemplary, and any radius could be selected as appropriate for andapplication. Alternatively, strictly a radius does not have to be used,but instead the at least one processor 301 could generate secondportions based on venues within specified geographic regions, such asfor example the city limits of the current city which the user islocated. The at least one processor 301 could also determine a range inwhich to search for venues based on a nature or type of the venue forwhich the using is searching. For example, if the at least one processor301 is looking for restaurants, the at least one processor 301 couldlimit the search area to a one-kilometer radius around the user. On theother hand, if the at least one processor 301 is searching for concerts,the at least one processor 301 could limit the search area to the citylimits of the city which the user is presently located or in which theuser is scheduled to be located (e.g., calendared appointments,electronic ticketing) during the period in question. Data regarding thevenues which the at least one processor 301 searches through can bestored on a non-transitory processor-readable storage medium 302 onsystem 300, or can be stored on a remote server accessible to the atleast one processor 301 of system 300.

Further, the above implementations may also take a user's previousengagements and venue choices into account when determining what venuesand events to suggest. For example, if the user regularly eatsMediterranean food, the at least one processor 301 could search for andsuggest Mediterranean restaurants around a user's current location.Additionally, the above implementations may also take time of day intoaccount when determining what venues and events to suggest. As anexample, the at least one processor 301 may be more likely to suggestbreakfast and coffee cafes in the morning, fast-food diners or take-outaround midday for lunch, upscale restaurants in the evening for dinner,and bars or shows at nighttime. Similarly, the at least one processor301 may limit restaurant and food venue suggestions to be only aroundthe user's typical eating hours.

FIG. 16 is a flowchart which illustrates another exemplary method 1600in accordance with the present systems, devices, and methods. Althoughmethod 1600 is described with reference to the system 300 shown in FIG.3 , method 1600 is equally applicable to any hardware arrangement havingat least one processor, a non-transitory processor-readable storagemedium, a wireless communications module, an output device, and a userinput mechanism. Method 1600 can be embodied as a set ofprocessor-executable instructions stored on the non-transitoryprocessor-readable storage medium 302, which can be executed by the atleast one processor 301. Method 1600 includes acts 1601, 1602, 1603,1604, and 1605, though those of skill in the art will appreciate that inalternative implementations certain acts may be omitted and/oradditional acts may be added. Those of skill in the art will alsoappreciate that the illustrated order of the acts is shown for exemplarypurposes only and may change in alternative implementations.

At 1601, at least one camera of system 300 captures visual data. At1602, the at least one processor 301 generates at least one candidatemessage based on the captured visual data. At 1603, the output device304 presents the at least one candidate message to the user. At 1604,the at least one processor 301 receives an input from the userindicative of a selected message from the at least one candidatemessage. At 1605, the wireless communications module 303 transmits theselected message from the first system 300, to be received by one or aplurality of other systems or devices.

The acts of method 1600 can be similar in at least some respects to theacts of method 400 in FIG. 4 and method 700 in FIG. 7 , and thus theacts of method 1600 in FIG. 16 can utilize any of the data sources andsensors, as well as the techniques, discussed above with regards togenerating at least one candidate message based on at least theenvironmental context parameter and/or an intended recipient (ifrecipient is determined first) in the implementations shown in FIGS. 4and 7 . For example, the environmental context can be determined atleast in part based on the output from any one of or any combination ofthe following: a clock, a calendar accessible by system 300, a locationsensor, a camera, a microphone, an electromyography sensor, amechanomyography sensor, an accelerometer, a gyroscope, anelectrocardiography sensor, a photoplethysmography sensor, a bloodpressure sensor, a location sensor, a global positioning system sensor,a compass, and an eye-tracking sensor. Further, the sensor(s) could becarried in system 300, or could be carried by a peripheral devicecommunicatively coupled to system 300.

Specific implementations of method 1600 are discussed in more detailbelow with reference to FIGS. 18-24 .

FIG. 17 is a flowchart which illustrates an exemplary method 1700 inaccordance with the present disclosure. Although method 1700 isdescribed with reference to the system 300 shown in FIG. 3 , method 1700is equally applicable to any hardware arrangement having at least oneprocessor, a non-transitory processor-readable storage medium, awireless communications module, an output device, and a user inputmechanism. Method 1700 can be embodied as a set of processor-executableinstructions stored on the non-transitory processor-readable storagemedium 302, which can be executed by the at least one processor 301.Method 1700 includes acts 1701, 1702, 1703, 1704, 1705, 1706, 1707, and1708, though those of skill in the art will appreciate that inalternative implementations certain acts may be omitted and/oradditional acts may be added. Those of skill in the art will alsoappreciate that the illustrated order of the acts is shown for exemplarypurposes only and may change in alternative implementations.

Method 1700 is similar to method 1600 but adds additional acts.Specifically, acts 1701, 1702, 1703, 1704, and 1708 of method 1700correspond to acts 1601, 1602, 1603, 1604, and 1605 of method 1600,respectively, and thus the disclosure regarding acts 1601, 1602, 1603,1604, and 1605 fully applies to method 1700 and the discussion will notbe repeated here in the interest of brevity. The additional acts 1705,1706, and 1707 focus on determining a recipient for the message andsending the message to the determined or intended recipient.Specifically, at 1705, the at least one processor 301 determines atleast one candidate recipient for the message based on at least one of acontent of the message and the captured visual data. At 1706, the outputdevice presents or displays the at least one candidate recipient for thecomplete message. At 1707, the at least one processor 301 receives asecond input from the user of the system 300, the second inputindicative of a selected recipient from the at least one candidaterecipient. At 1708, the wireless communications module transmits thecomplete message from system 300 to the selected recipient.

These additional acts 1705, 1706, and 1707 may be similar to acts 705,706, and 707 in method 700, and thus acts 1705, 1706, and 1707 canutilize any of the data sources and sensors, as well as the techniques,discussed above with regards to determining at least one candidaterecipient based on at least the environmental context parameter and/orthe content of the message in the implementation shown in FIG. 7 . Forexample, the environmental context can be determined at least in partbased on the output from any one of or any combination of the following:a clock, a calendar accessible by system 300, a location sensor, acamera, a microphone, an electromyography sensor, a mechanomyographysensor, an accelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a location sensor,a global positioning system sensor, a compass, and an eye-trackingsensor. Further, the sensor(s) could be carried in system 300, or couldbe carried by a peripheral device communicatively coupled to system 300.

As discussed above regarding FIG. 1B, although it is possible to performall of the acts of method 1600 and method 1700 on a single device, it isalso possible to remove some of the hardware from a presentation device(such as a WHUD), and utilize a peripheral device comprising saidhardware instead, in order to reduce bulk in the presentation device. Asan example, in the case of methods 1600 and 1700, acts 1601 and 1701could be performed by a camera of a WHUD, acts 1602, 1604, 1702, 1704,1705, and 1707 could be performed by a processor of the WHUD, acts 1603,1703, and 1706 could be performed by an output device of the WHUD, andacts 1605 and 1708 could be performed by a wireless communicationsmodule of a peripheral device communicatively coupled to the WHUD, suchthat the processing and presenting acts are performed on the WHUD, andthe peripheral device is used as a wireless transmission gateway.Alternatively, acts 1601 and 1701 could be performed by a camera of aWHUD, acts 1602, 1604, 1702, 1704, 1705, and 1707 could be performed bya processor of a peripheral device communicatively coupled to the WHUD,acts 1603, 1703, and 1706 could be performed by an output device of theWHUD, and acts 1605 and 1708 could be performed by a wirelesscommunications module of the peripheral device, such that the peripheraldevice performs processing and transmission, whereas the WHUD performsdata collection and output presentation.

FIGS. 18A-18D represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 18A, proceeding to FIG. 18B, proceeding to FIG.18C, proceeding to FIG. 18D. In these Figures, a user decides that theywant to capture visual data, so they select a “Camera Capture” option oruser selectable icon presented to the user (e.g., presented on theoutput device 304 of system 300) as illustrated in FIG. 18A. Thisinterface is exemplary, and could be replaced for example by a shortcutto a camera capture mode, such as a hardware camera button. System 300can then enter a camera capture mode as shown in FIG. 18B, enabling theuser to capture visual data. In the case of FIG. 18B, the user cancapture visual data representing a chalkboard or whiteboard on whichnotes were taken during a meeting.

The at least one processor 301 of system 300 can then use at least oneautomated image analysis algorithm to analyze the captured visual dataand autonomously generate at least one (preferably multiple) candidatemessages for the user to choose from based on at least the capturedvisual data, and present the at least one generated message to the useras shown in FIG. 18C. Optionally, the at least one generated message canalso be based on an environmental context parameter of the system 300.In the case of FIG. 18C, the at least one processor 301 can analyze thecaptured visual data to identify the chalkboard or whiteboard, which arecommonly used to take notes and discuss ideas. Consequently, the atleast one processor 301 can generate at least one candidate messagewhich pertains to the concept of sharing meeting notes. For example, theat least one processor 301 may access at least one of a clock and/orcalendar accessible to system 300, to generate a candidate message whichindicates at least one of a time, date, location, topic, or otherrelevant information. In the example shown in FIG. 18C, the at least oneprocessor 301 could access a clock of system 300 to assign a time and/ordate stamp to the message, and the at least one processor 301 couldaccess a calendar to determine that the user was/is in a meeting withthe Hardware Team at the time of capturing the visual data or shortlythereafter. Using this information, the at least one processor 301 cangenerate a message like “Jun. 10, 2019 Hardware Team meeting notes”. Asanother example, the at least one processor 301 could identify thepresence of the chalkboard or the whiteboard in the captured visualdata, and could thus generate messages which pertain to solicitingdialogue. For example, the at least one processor 301 could generate acandidate message like “Any thoughts on this?” as shown in FIG. 18C. Asshown in FIG. 18C, the at least one candidate message can be presentedto the user by output device 304, optionally alongside a preview 1810 ofthe captured visual data.

The system 300 can then receive an input from the user indicating aselection of a message of the at least one candidate message. The atleast one processor 301 can then determine at least one candidaterecipient for the selected message based on at least one of the capturedvisual data, content of the selected message, and at least oneenvironmental context parameter of the system 300. A candidate recipientfor a message may be determined autonomously by the at least oneprocessor 301 based on at least one of the captured visual data, theenvironmental context parameter, and/or the content of the message.Determining the recipient based on environmental context parameter canuse any of the hardware, sensors, and techniques discussed aboveregarding determining the environmental context parameter. For example,the environmental context can be determined at least in part based onthe output from any one of or any combination of the following: a clock,a calendar accessible by system 300, a location sensor, a camera, amicrophone, an electromyography sensor, a mechanomyography sensor, anaccelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a globalpositioning system sensor, a compass, and an eye-tracking sensor.Further, the sensor(s) could be carried by the system 300, or could becarried by an external device communicatively coupled to system 300.Additionally or alternatively, determining a recipient for eachcandidate message can be based on the content of the respectivecandidate message. For example, if a message says “Jun. 10, 2019Hardware Team Meeting Notes”, the at least one processor 301 maydetermine that candidate recipients of the message should be people whoattended the Hardware Team meeting, or people related to the HardwareTeam, such as the user's boss Jean, as illustrated in FIG. 18D.

FIG. 18D also illustrates an important concept: that a “message” in thecontext of the present systems, devices and methods is not limited to amessage which is sent from one user to another user. Instead, a“message” can also include content submissions, such as to content hubswhich are not tied to a single user. In the example of FIG. 18D, the atleast one processor 301 could determine that since the visual data wascaptured during or shortly after the Hardware Team meeting, the messagecould be shared to the Hardware Team drive, such as a cloud-based drivestored on a remote server. This is illustrated in FIG. 18D by acandidate recipient of “Share to Hardware Team Drive”. As anotherexample, the at least one processor 301 could determine that since thevisual data includes a representation of a chalkboard or whiteboard, thevisual data likely contains notes that the user wishes to retain, andthus may determine a user's personal drive as an appropriate recipientof the message. This is illustrated in FIG. 18D as a candidate recipientof “Store on personal drive”. Such a personal drive could be acloud-based drive on a remote server, or could be a local drive insystem 300.

The user can then select a desired recipient or recipients, and themessage, including the captured visual data and the text generated bythe at least one processor 301, can be transmitted by the wirelesscommunications module 303, to be received by the desired recipient.

Similar to the examples illustrated in at least FIGS. 5A, 5B, and 6above, the user interface in FIG. 18A-18D could be made more compact byjoining multiple steps taken by the user together. For example, the usermay be able to select both a message to be sent and a recipient of themessage with a single input. In such an example, the at least oneprocessor 301 can generate at least one candidate message, pair the atleast one candidate message with a recipient, and output device 304 canoutput the pair of candidate message and recipient. The user may simplyselect the paired message and recipient, quickly and efficiently.

FIGS. 19A-19D represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 19A, proceeding to FIG. 19B, proceeding to FIG.19C, proceeding to FIG. 19D. In these Figures, a user decides that theywant to capture visual data, so they select a “Camera Capture” option oruser selectable icon presented to the user (e.g., shown on the outputdevice 304 of system 300) as illustrated in FIG. 19A. This interface isexemplary, and could be replaced for example by a shortcut to a cameracapture mode, such as a hardware camera button. System 300 can thenenter a camera capture mode as shown in FIG. 19B, enabling the user tocapture visual data. In the case of FIG. 19B, the user can capturevisual data representing people.

The at least one processor 301 of system 300 can then use at least oneautomated image analysis algorithm to analyze the captured visual dataand autonomously generate at least one (preferably multiple) candidatemessages for the user to choose from based on at least the capturedvisual data, and output device 304 can present the at least onegenerated message to the user as shown in FIG. 19C, optionally alongsidea preview 1910 of the captured visual data. Optionally, the at least onegenerated message can also be based on an environmental contextparameter of the system 300. In the case of FIG. 19C, the at least oneprocessor 301 can analyze the captured visual data to identify that thecaptured visual data includes representations of people. The at leastone processor 301 can also usual a facial recognition algorithm todetermine the identities of the people in the picture, and could alsouse an algorithm which analyzes the emotional expression of the peoplein the picture. The at least one processor 301 can then generate atleast one candidate message which pertains to the people in the capturedvisual data. In the example shown in FIG. 19C, the at least oneprocessor 301 could determine that the people in the picture are theuser's Uncle George and Aunt Tina, and could generate a message whichincludes this information, such as “Having fun with Uncle George andAunt Tina!”. If the at least one processor 301 analyzes the emotionalexpression of the people in the picture, this information can be used togenerate more accurate content. In the example of FIG. 19C, the at leastone processor 301 could determine that both Uncle George and Aunt Tinaare smiling, and thus everyone in the picture is “Having Fun”. Further,the at least one processor could determine that the user is smiling orhappy by analyzing emotional expression of the user's eyes using an eyetracking sensor such as eye-tracking system 184, and/or by analyzingemotional expression of the user's eyebrows or cheeks using facialexpression detection sensors such as proximity sensors 185. Accordingly,the at least one processor 301 can generate a candidate message like“Having fun with Uncle George and Aunt Tina”.

The system 300 can then receive an input from the user indicating aselection of a message of the at least one candidate message. The atleast one processor 301 can then autonomously determine at least onecandidate recipient for the selected message based on at least one ofthe captured visual data, the content of the selected message, and atleast one environmental context parameter of the system 300. Determiningthe recipient based on environmental context parameter can use any ofthe hardware, sensors, and techniques discussed above regardingdetermining the environmental context parameter. For example, theenvironmental context can be determined at least in part based on theoutput from any one of or any combination of the following: a clock, acalendar accessible by system 300, a location sensor, a camera, amicrophone, an electromyography sensor, a mechanomyography sensor, anaccelerometer, a gyroscope, an electrocardiography sensor, aphotoplethysmography sensor, a blood pressure sensor, a globalpositioning system sensor, a compass, and an eye-tracking sensor.Further, the sensor(s) could be carried by the system 300, or could becarried by an external device communicatively coupled to system 300.

Additionally or alternatively, determining a recipient for eachcandidate message can be based on the content of the respectivecandidate message. For example, if a message says “Having fun with UncleGeorge and Aunt Tina”, the at least one processor 301 may determine thatcandidate recipients of the message should be people who would knowUncle George and Aunt Tina, such as members of the family of the user.For example, the at least one processor 301 could determine that theuser's Dad, or the user's husband Bert are appropriate candidaterecipients for the message, as illustrated in FIG. 19D as candidaterecipients “Dad” and “Husband Bert”. Additionally, the message itselfrefers to Uncle George and Aunt Tina, which would be unusual content ifthe message were to be sent to Uncle George or Aunt Tina. Thus, the atleast one processor 301 may exclude Uncle George and Aunt Tina asrecipients for the message if the content of the message would beunusual to send to them.

Similar to FIG. 18D above, FIG. 19D also illustrates the concept that a“message” can also include a content submission, such as to a contenthub which is not tied to a single user. In the example of FIG. 19D, theat least one processor 301 could determine that the content of thecaptured visual data and the generated text for the message are suitablefor public sharing, such as to a social network like Facebook®.Consequently, system 300 may present “Upload to social media” as acandidate recipient for the message, as shown in FIG. 19D. Further, asubsequent interface can be presented which enables the user to selectwhich social network the message should be shared to, in the even theuser is a member of multiple social networks.

The user can then select a desired recipient or recipients, and themessage, including the captured visual data and the text generated bythe at least one processor 301, can be transmitted by the wirelesscommunications module 303, to be received by the desired recipient.

Similar to the examples illustrated in at least FIGS. 5A, 5B, and 6above, the user interface in FIGS. 19A-19D could be made more compact byjoining multiple steps taken by the user together. For example, the usermay be able to select both a message to be sent and a recipient of themessage with a single input. In such an example, the at least oneprocessor 301 can generate at least one candidate message, pair the atleast one candidate message with a recipient, and the output device 304can output the pair of the candidate message and recipient. The user maysimply select the paired message and recipient, quickly and efficiently.

FIGS. 20A-20D represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 20A, proceeding to FIG. 20B, proceeding toFigure proceeding to FIG. 20D. FIGS. 20A and 20B are similar to FIGS.19A and 19B discussed above, and thus the description of FIGS. 19A and19B is also applicable to FIGS. 20A and 20B.

In FIG. 20C, the at least one processor 301 may determine that since thecaptured visual data contains representations of people, the user maywish to share the captured visual data with the people in the picture.To this end, the at least one processor 301 could generate a simplemessage like “Here is today's picture.” as illustrated in FIG. 20C. Asshown in FIG. 20D, the at least one processor 301 can determinecandidate recipients which include people identified in the capturedvisual data, in this case “Uncle George” and “Aunt Tina”, as illustratedin FIG. 20D presented by output device 304. Further, the user may notwish to share the picture, but may instead simply want to store it forlater review, and so the at least one processor 301 may also identifydata storage, such as the user's cloud storage drive, as being acandidate recipient for the message, as illustrated in FIG. 20D ascandidate recipient “Upload to cloud storage”.

The user can then select a desired recipient or recipients, and themessage, including the captured visual data and the text generated bythe at least one processor 301, can be transmitted by the wirelesscommunications module 303, to be received by the desired recipient.

Similar to the examples illustrated in at least FIGS. 5A, 5B, and 6above, the user interface in FIG. 20A-20D could be made more compact byjoining multiple steps taken by the user together. For example, the usermay be able to select both a message to be sent and a recipient of themessage with a single input. In such an example, system 300 can generateat least one candidate message, pair the at least one candidate messagewith a recipient, and output the pair on output device 304 of system300. The user may simply select the paired message and recipient,quickly and efficiently. As an example, with reference to the case ofFIGS. 20A-20D, the at least one processor 301 may pair the messagecontent “Here is today's picture” with the recipients “Uncle George” and“Aunt Tina”.

FIGS. 21A-21E represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 21A, proceeding to FIG. 21B, proceeding to FIG.21C, proceeding to FIG. 21D, proceeding to FIG. 21E. FIGS. 21A and 21Bare similar to FIGS. 19A and 19B discussed above, and thus thedescription of FIGS. 19A and 19B is also applicable to FIGS. 21A and21B.

In FIG. 21C, system 300 may present an option for the user to customizeor personalize content of a message, such as to provide a more personalor “human” feel to the message. This is illustrated by the “Personalize”option illustrated in FIG. 21C. FIG. 21D shows an exemplary interfacefor customizing the message. At least some of the content may beautonomously generated by the at least one processor 301, and the usermay be presented with the ability to edit or customize the content. Forexample, as shown in FIG. 21D, the at least one processor 301 maygenerate a caption like “Great day with Uncle George and Aunt Tina atthe Calgary Zoo!”. Such a message could be based on analysis of thecaptured visual data, such as for example a facial recognition algorithmidentifying people in the image, and could also be based on anenvironmental context parameter of system 300 obtained using any of thetechniques discussed herein. In the above example, a location of system300 can be obtained from a location sensor, which can then be used todetermine content for the message, in this case that the user capturedthe visual data at the Calgary Zoo. Further, an interface can beprovided which enables the user to customize the generated content. Forexample, the user may be able to select words or phrases in the textcontent, and select alternative words or phrases. As one example, theuser could select “Great day”, and the system could present alternativephrases like “Had fun”, “Fun family day”, or similar. Additionally, aninterface could be presented for personalizing the captured visual dataas well. In the example of FIG. 21D, an interface could be presentedwhich allows the user to apply stamps, emojis, catchphrases, frames,filters, and more to the captured image, alongside a preview 2110 of thecustomized visual data. For example, the user could select a flowerstamp from a customization panel 2120, and then could click on anynumber of points of the captured visual data to apply the flower stampthereto. A similar procedure could be performed for emojis orcatchphrases, such as “So much fun!” and “Happy times” as shown in FIG.21D. Filters and color adjustment could also be presented which, whenselected by the user, apply a filter or color adjustment to the image.The stamps, emojis, catchphrases, frames, filters and othercustomization options presented to the user can be automaticallydetermined, such as for example presenting the most popular options tothe user, or options which the user themselves frequently use. Further,the options presented could also be based on analysis of the capturedvisual data and/or an environmental context parameter of system 300. Forexample, if the at least one processor analyzes the captured visual dataand determines that the captured visual data is too dark, the at leastone processor 301 can suggest applying brightening filters to thecaptured visual data. Additionally, the at least one processor 301 coulddetermine a time or date when the visual data was captured, and suggestappropriate options based on this time or date. For example, if thevisual data was captured on or around October 31, Halloween, the system300 could present spooky or pumpkin themed emojis. Further, system 300could autonomously decorate the captured visual data using the aboveoptions, and present the user the option to customize the decorations,such as by moving or changing stamps or other decorations. Further, thesystem 300 could present multiple autonomously generated decoratedand/or captioned messages, and allow the user to select a desiredmessage, which could optionally be further customized by the user.

The system 300 can receive an input from the user indicating a selectionor completion of a customized message. The at least one processor 301can then autonomously determine at least one candidate recipient for theselected message as shown in FIG. 21E based on at least one of thecaptured visual data, the content of the selected message, and at leastone environmental context parameter of the system 300. Determining therecipient based on environmental context parameter can use any of thehardware, sensors, and techniques discussed above regarding determiningthe environmental context parameter. For example, the environmentalcontext can be determined at least in part based on the output from anyone of or any combination of the following: a clock, a calendaraccessible by system 300, a location sensor, a camera, a microphone, anelectromyography sensor, a mechanomyography sensor, an accelerometer, agyroscope, an electrocardiography sensor, a photoplethysmography sensor,a blood pressure sensor, a global positioning system sensor, a compass,and an eye-tracking sensor. Further, the sensor(s) could be carried bythe system 300, or could be carried by an external devicecommunicatively coupled to system 300.

Additionally or alternatively, determining a recipient for eachcandidate message can be based on the content of the respectivecandidate message. For example, if a message says “Great day with UncleGeorge and Aunt Tina at the Calgary Zoo”, the at least one processor 301may determine that candidate recipients of the message should be peoplewho would know Uncle George and Aunt Tina, such as members of the familyof the user. For example, the at least one processor 301 could determinethat the user's Dad, or the user's husband Bert are appropriatecandidate recipients for the message, illustrated as candidaterecipients “Dad” and “Husband Bert” in FIG. 21E.

Similar to FIG. 18D above, FIG. 21E also illustrates the concept that a“message” can also include a content submission, such as to a contenthub which is not tied to a single user. In the example of FIG. 21E, theat least one processor 301 could determine that since the messagecontent is personalized with stamps or similar, the at least oneprocessor can determine that the message is suitable to share on asocial network like Facebook®. Consequently, system 300 may presentsocial media as a candidate recipient for the message, as shown in FIG.21E.

The user can then select a desired recipient or recipients, and themessage, including the captured visual data, the text (caption)generated by the at least one processor 301, and any customization orpersonalization can be transmitted by the wireless communications module303, to be received by the desired recipient.

FIGS. 22A-22D represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 22A, proceeding to FIG. 22B, proceeding to FIG.22C, proceeding to FIG. 22D. In these Figures, a user decides that theywant to capture visual data, so they select a “Camera Capture” option oruser selectable icon presented to the user (e.g., shown on the outputdevice 304 of system 300) as illustrated in FIG. 22A. This interface isexemplary, and could be replaced for example by a shortcut to a cameracapture mode, such as a hardware camera button. System 300 can thenenter a camera capture mode as shown in FIG. 22B, enabling the user tocapture visual data. In the case of FIG. 22B, the user can capturevisual data representing food.

The at least one processor 301 can then use at least one automated imageanalysis algorithm to analyze the captured visual data, and determineappropriate action options to present to the user based on the analysis.In the example of FIG. 22C, the at least one processor 301 may identifythat the captured visual data includes a representation of food, and mayautonomously generate action options for the user, including options tosubmit a review of the food, or save the captured visual data to submita review later, once the food has been eaten. Other options couldinclude sharing the captured visual data to social media or sending thecaptured visual data to a contact, similar to the implementationsdiscussed above.

If the user selects to share a review, the at least one processor 301may autonomously generate at least one (preferably multiple) candidatemessages for the user to choose from, and present the at least onecandidate message to the user as shown in FIG. 22D. In the case of FIG.22D, the at least one processor 301 can generate several “star” ratings,such as 1 star, 2 star, 3 star, 4 star, and 5 star, as shown by staricons/emojis in FIG. 22D. The at least one processor 301 can alsoautonomously generate message content appropriate to each star rating,which can be selected and optionally customized by the user. Thegenerated message content could be based on generic review templates,the captured visual data, or an environmental context parameter of thesystem 300, as examples. For example, the at least one processor 301could use generic review content, like “Amazing” or “A good date spot”as shown in FIG. 22D. As another example, the at least one processor 301could analyze the content of the captured visual data to identifyobjects represented in the data, and generate message content based onthe identified objects, such as “Soggy fries” or “Meat was rotten” asshown in FIG. 22D. If the object detection is sufficientlysophisticated, the at least one processor 301 could make a determinationof the food quality based on the appearance of the food. For example, ifthe fries have a soggy appearance, the at least one processor 301 couldgenerate message content for low-star ratings like “Soggy fries”.Further, even if the at least one processor 301 cannot accuratelyidentify specific objects in the captured visual data, the at least oneprocessor 301 can use at least one environmental context parameter toinfer what the user might be eating, and generate corresponding messagecontent. For example, if a location sensor of system 300 indicates thatthe user is at a steakhouse, the at least one processor 301 couldgenerate review content which makes reference to items typically servedat a steakhouse, such as meat. Further, if a menu for the restaurant atwhich the user is located is available to system 300, such as on awebsite of the restaurant, the at least one processor 301 may generatemessage content based on popular menu items or menu themes. As examplesof the above, the at least one processor 301 could determine that if theuser is dining at “Friendly Steakhouse”, the user is likely eating meat.This could be further reinforced if the at least one processor 301analyzes the menu for Friendly Steakhouse and determines that there areno vegetarian entrees. Consequently, that at least one processor 301could generate message content like “Meat was rotten” as shown in FIG.22D.

If the user selects an option such as “Save and review later” shown inFIG. 22C, a similar process for generating message content can beperformed when the user later decides to complete the review.

The system 300 can then receive an input from the user indicating aselection of a message of the at least one candidate message. The atleast one processor 301 can then autonomously determine at least onecandidate recipient for the selected message based on at least one ofthe captured visual data, the content of the selected message, and atleast one environmental context parameter of the system 300. Forexample, if available, the at least one processor 301 may identify therestaurant itself maintains a review board or submission address, andthus may identify such a board or address as a candidate recipient forthe review. Alternatively, the at least one processor may identifyaggregate review sites such as TripAdvisor® or Yelp® as candidaterecipients for the review.

The user can then select a desired recipient or recipients, and themessage, including the review content generated by the at least oneprocessor 301, and optionally the captured visual data, can betransmitted by the wireless communications module 303, to be received bythe desired recipient.

Similar to the examples illustrated in at least FIGS. 5A, 5B, and 6above, the user interface in FIGS. 22A-22D could be made more compact byjoining multiple steps taken by the user together. For example,selection of the “Share review” option by the user may also be pairedwith a recipient of the review, such as for example a “Share review toTripAdvisor” option.

FIGS. 23A-23D represent a sequence of displays or presentations of userinterfaces including messages which can be shown or presented to theuser, starting from FIG. 23A, proceeding to FIG. 23B, proceeding to FIG.23C, proceeding to FIG. 23D. FIGS. 23A, 23B, and 23C are similar toFIGS. 22A, 22B, and 22C discussed above, and thus the description ofFIGS. 22A, 22B, and 22C is also applicable to FIGS. 23A, 23B, and 23C.One difference between FIG. 23C and FIG. 22C is that in FIG. 23C, theuser selects the “Upload to social media” option or the “Send tocontact” option.

In FIG. 23D, the at least one processor 301 may generate candidatemessages based on at least one of the captured visual data, anenvironmental context parameter of the system 300. For example, the atleast one processor 301 can analyze the captured visual data to identifya representation of Steak Frites in the captured visual data, andgenerate appropriate candidate messages based on this identification,such as “Steak trites are the best!” or “Meat eater's delight!” as shownin FIG. 23D. Even if the at least one processor 301 is not able toidentify specific food objects in the captured visual data, the at leastone processor 301 may identify that the captured visual data includes arepresentation of food in general. The at least one processor 301 mayconsequently generate a food-based candidate message like “Yummy!” asshown in FIG. 23D. Further, any of the generated candidate messages mayalso include emojis generated or retrieved by the at least one processor301, such as for example the star-eyed emoji with a protruding tongue asshown in FIG. 23D.

The at least one processor 301 may receive an input from the userindicating a selection of a candidate message, and the at least oneprocessor 301 may determine at least one candidate recipient for themessage. Such candidate recipients could be determined using any of theexemplary techniques discussed herein. For example, if the user selectedthat the “Upload to social media” option in FIG. 23C, the candidaterecipients may be different social media networks which the user is amember of. As another example, if the user selected the “Send tocontact” option in FIG. 23C, the at least one processor 301 may identifysome close friends or family of the user who may be interested infood-related messages.

The user can then select a desired recipient or recipients, and themessage, including the captured visual data and the text generated bythe at least one processor 301, can be transmitted by the wirelesscommunications module 303, to be received by the desired recipient.

Similar to the examples illustrated in at least FIGS. 5A, 5B, and 6above, the user interface in FIG. 23A-23D could be made more compact byjoining multiple steps taken by the user together. For example, the“Upload to social media” option may instead be an “Upload to Facebook”option, which can be selected with a single input from the user andindicates both the desired action and the intended recipient.

In many of the above discussed implementations, a user first selects“Camera Capture”, captures visual data, and then candidate messages andcandidate recipients are determined. However, this order of operationscould be changed, as illustrated by the implementations shown in FIGS.24A-24D. FIGS. 24A-24D represent a sequence of displays or presentationsof user interfaces including messages which can be shown or presented tothe user, starting from FIG. 24A, proceeding to FIG. 24B, proceeding toFIG. 24C, proceeding to FIG. 24D.

In FIG. 24A, a user decides that they want to send a message, and thusselects the “Compose Message” option. In FIG. 24B, the at least oneprocessor 301 generates at least one candidate message based on anenvironmental context parameter of the device. The at least oneprocessor 301 can generate the at least one candidate message using anyof the techniques discussed herein, based on environmental context datafrom any of the sensors discussed herein. In the example of FIG. 24B,data from a location sensor of system 300 can indicate that the user isat “Friendly Steakhouse”, and can generate candidate messages related tothis location, such as “I'm at Friendly Steakhouse” or “Trying a newsteak restaurant” as shown in FIG. 24B.

The user can select one of the candidate messages, and the system 300can present an option for the user to share the selected message, or toshare the selected message with a capture, as shown in FIG. 24C. If theuser selects “Share with capture”, a camera capture interface can bepresented, which allows the user to capture visual data to be sent withthe message. The at least one processor 301 can then determine at leastone candidate recipient for the message using any of the techniques fordetermining candidate recipients discussed herein, and receive an inputfrom a user indicating a selection of a recipient of the message.Subsequently, system 300 can send the message with the captured visualdata to the selected recipient. If the user selects the “Share” optionin FIG. 24C, the camera capture step of FIG. 24D can be skipped.

FIGS. 25A-25C represent a sequence of displays or presentations of userinterfaces for generating a message to share, starting from FIG. 25A,proceeding to FIG. 25B, proceeding to FIG. 25C.

In FIGS. 25A-25C, a user decides that they want to capture visual data,so they select a “Camera Capture” option or user selectable iconpresented to the user (e.g., shown on the output device 304 of system300) as illustrated in FIG. 25A. This interface is exemplary, and couldbe replaced for example by a shortcut to a camera capture mode, such asa hardware camera button. System 300 can then enter a camera capturemode as shown in FIG. 25B, enabling the user to capture visual data. Inthe example of FIG. 25B, the at least one processor 301 can infer theintent of the user based on the content visible to the camera and/or atleast one environmental context parameter. In the example of FIG. 25B, alocation sensor of system 300 could indicate that the user is in agrocery store. Further, the at least one processor 301 could analyzevisual data captured by the camera to determine that the user is holdingan object in their hand (a can of soda pop 2540 in FIG. 25B) to be thesubject of the captured visual data. From this, the at least oneprocessor 301 can infer that the user is considering purchasing the helditem, but wants to confirm with another person whether the purchase isappropriate. Based on this inference, the at least one processor 301 cangenerate a candidate message. In the example FIG. 25B, the at least oneprocessor 301 can do this in a streamlined, visual manner, where emojis2550 are overlayed on the captured visual data, and the emojisthemselves represent the content of the message. In FIG. 25B, this isillustrated by a smiley-face emoji, a question mark, and adisgusted-face emoji, which implies a question of “Is this good orbad?”, or “Do you want this?”.

Further, the capture process illustrated in FIG. 25B can be carried outin different ways. In a first implementation, the at least one processor301 can be analyzing visual data from the camera before the useractually selects to capture the visual data. For example, the at leastone processor 301 could analyze a video feed from the camera before theuser clicks a button to take a photograph. Based on this analysis, theat least one processor 301 could generate the message content shown inFIG. 25B (the emojis 2550 and question mark), and output device 304could output this content to the user before the user has even clickedto take a photograph. This effectively allows the user to preview thegenerated message content before capturing an image. After system 300receives user input to capture an image, the generated content could beautomatically added to the image as it was presented to the user, andthe at least one processor 301 could generate candidate recipients forthe message based on the captured visual data and/or an environmentalcontext parameter of the device, such that the output device 304 canimmediately present a recipient selection interface to the user. Fromthe user's perspective, this could result in a smooth flow where theyselect camera capture, capture visual data which automatically includesthe desired content, and then proceed to select a recipient.

In a second implementation, the user can capture visual data, such as byclicking to take a photograph, and the at least one processor 301 canthen analyze the captured visual data to generate candidate messagecontent. The message content can then be overlayed on the capturedvisual data, and presented by output device 304 as a candidate messageto the user for confirmation/selection. The system 300 cansimultaneously or subsequently present at least one candidate recipientto the user for selection. This second implementation is less smooth andrequires more steps from the user, but will advantageously reduce theprocessing required, and thus will reduce power consumption.

In some implementations, at least one candidate recipient can bepresented to the user for selection simultaneously with selection of thecandidate message, for more streamlined operation. For example, in thefirst implementation above, a candidate recipient could be presented tothe user in FIG. 25B before the user provides input to capture aphotograph. As another example, in the second implementation above, acandidate recipient could be presented to the user simultaneously withthe candidate message, such that the user need only make one selectionfor both the message and the recipient.

Determining at least one candidate recipient in the aboveimplementations of FIGS. 25A-25C can be performed using any of thetechniques for determining candidate recipients discussed herein, andcan be based on data from any of the sensors for acquiring environmentalcontext parameters described herein. For example, the at least oneprocessor 301 could determine that since the user is at a grocery storeand is asking about a product to buy, candidate recipients will likelybe people living in the same household as the user, such as the user'sfamily. FIG. 25C illustrates candidate recipients as “Dad”, “HusbandBert”, and “Daughter Mildred” for example. As another example, the atleast one processor 301 could access a calendar of the user, anddetermine that the user is attending a party later in the evening.Consequently, the at least one processor 301 may identify the host ofthe party as a candidate recipient.

FIGS. 26A-26C represent a sequence of displays or presentations of userinterfaces for generating a message to share, starting from FIG. 26A,proceeding to FIG. 26B, proceeding to FIG. 26C.

In FIGS. 26A-26C, a user decides that they want to capture visual data,so they select a “Camera Capture” option or user selectable iconpresented to the user (e.g., shown on the output device 304 of system300) as illustrated in FIG. 26A. This interface is exemplary, and couldbe replaced for example by a shortcut to a camera capture mode, such asa hardware camera button. System 300 can then enter a camera capturemode as shown in FIG. 26B, enabling the user to capture visual data.

The user can then perform a gesture in front of the camera of system300, for conversion to at least one emoji. In the example of FIG. 26B,the user can perform a thumbs-up gesture in front of the camera. Thecamera can capture visual data of this gesture, and the at least oneprocessor 301 can generate an emoji message based on this gesture.

Generation of the message could be performed in multiple ways. In afirst implementation, the at least one processor 301 could analyze thecaptured image data to match the gesture performed by the user to adatabase of gestures. In practice, such a database could include a greatnumber of gestures, such as an “OK” gesture, a “thumbs-down” gesture, a“devil-horns” gesture, or many more. The at least one processor 301could then retrieve an emoji corresponding to the gesture that the useris performing. In a second implementation, the at least one processor301 could make the emoji from the captured visual data. For example, theat least one processor 301 could recognize the bounds of the user's handor hands, and render an emoji which only includes the user's hands. Thiscould be achieved by the at least one processor removing datarepresenting parts of the captured visual data which do not representthe user's hands, and/or reducing size of the visual data, and/orreducing the number of colors represented in the visual data. Forexample, the captured visual data could be cropped to only include theuser's hands, and background data could be deleted or replaced with asolid background color. In this way, an emoji can be rendered based onthe captured visual data, which only includes a representation of theuser hand or hands for simplicity. This option would provide a flexiblesolution which can generate an emoji-based message even if the gestureperformed by the user does not match a gesture in a database.

It is also possible for the above gestures to be dynamic gestures, i.e.gestures which change over time. For example, “clapping” could beconsidered a dynamic gesture, where the user claps their hands together.If the at least one processor recognizes such a dynamic gesture, such asby analyzing multiple frames of captured visual data over time, the atleast one processor 301 can generated an animated emoji, eitherretrieving such and emoji from a database, or rendering a new one.Further, a dynamic gesture could also be represented by a static emoji,and does not necessarily require multiple animation frames. For example,for the clapping gesture, an emoji could include a single image of twohands together, with motion lines around the hands which imply that thehands are moving, even if the emoji itself does not contain animation ormovement.

Before sending the message, a sample of the emoji 2630 can be presentedto the user for confirmation, as illustrated in FIG. 26C. The at leastone processor 301 can also determine at least one candidate recipientfor the generated message using any of the techniques discussed herein,and any environmental context parameters from any of the sensorsdiscussed herein. The system 300 can receive input from the userindicating a selection of a recipient, and the message can betransmitted by wireless communications module 303 to be received by thedesired recipient.

The present systems, device, and methods may also infer an activity ofthe user based on partial objects viewable within a scene, instead ofrelying on identification of single whole objects. FIGS. 27, 28, and 29are exemplary scenes which illustrate this concept.

FIG. 27 illustrates an example scene 2700 as might be captured by acamera included on a WHUD system worn by a user operating a vehicle. Theat least one processor 301 included in the WHUD system can analyzevisual data captured by the camera to detect vehicle features, vehicleaccessories, or vehicle infrastructure represented in the visual datawhich indicate whether a user is operating a vehicle. For example, theat least one processor could identify equipment and features which canbe found in a vehicle, such as a steering wheel 2701 and at least oneuser hand 2702 thereon, at least one information indicator 2703,rear-view mirror 2704, side-view mirror 2705, display screen 2706,controls 2707, air vents 2708, sun visor 2709, dashboard 2710,frame-pillar 2711, roof 2712, or any other vehicle interior feature asappropriate. Information indicator 2703 could include for example aspeedometer, engine RPM gauge, engine temperature gauge, odometer, fuelgauge, status indication lights, warning lights, vehicle informationscreen, or any other indicator appropriate for a given application.Display screen 2706 could include for example a stereo display, aback-up camera display, a GPS or map display, an entertainment display,or any other display as appropriate. Controls 2707 could include atleast one user input interface for providing user control of anyfeatures of the vehicle, such as buttons, knobs, switches, levers,touch-sensitive inputs, or any other user interface as appropriate for agiven application. The at least one processor 301 may also identifyfeatures external to the vehicle which can indicate whether a user ofthe WHUD system is operating a vehicle, such as at least one othervehicle 2721, at least one road line 2722, or any other appropriateexternal feature. The at least one other vehicle 2721 could beidentified by identifying features of the at least one other vehicle2721, such as a general vehicular shape of the other vehicle, a licenseplate, windows, wheels, mirrors, at least one exhaust pipe, andindicator lights, or any other appropriate features. Indicator lights ofthe other vehicle could include taillights, brake lights, turn signals,back-up lights, headlights, or any other appropriate lights. One skilledin the art will appreciate that the present systems, devices, andmethods can utilize identification of the above described featuresexternal to the vehicle to determine whether the user is operating avehicle, regardless of the type of vehicle the user is driving. That is,features external to the vehicle are independent of the vehicle, andthus can be identified to determine whether a user is operating avehicle in any of the implementations discussed herein.

Further, identified features such as those listed above may not becompletely represented in the visual data. As an example, the capturedvisual data does not include a whole representation of the vehicle whichthe user is operating. As another example, the captured visual data onlypartially represents steering wheel 2701. As another example, thecaptured visual data only represents the user's hands 2702, and does notrepresent an entire body of the user. Nonetheless, the at least oneprocessor 301 can identify that the user is operating a vehicle based onidentification of these partially visible features.

After identification that the user is operating a vehicle, the at leastone processor 301 can generate at least one candidate message,optionally based on additional environmental context parameters of thedevice, using any of the techniques discussed herein, based on data fromany of the sensors discussed herein. The at least one processor 301 canalso determine at least one candidate recipient using any of thetechniques discussed herein, based on data from any of the sensorsherein. As an example, if the at least one processor 301 determines thatthe user is operating a vehicle, and the time is currently within a timerange that the user typically goes from a workplace to home, the atleast one processor may generate a candidate message like “Headinghome”, and a candidate recipient of the user's spouse. However, itshould be noted that there can be potential safety risks by enabling auser to operate a mobile device while operating a vehicle, especially ifsuch operation is taxing on the user's focus and concentration. This isdiscussed in more detail in U.S. Provisional Patent Application Ser. No.62/750,060. In view of this, it may be preferable to make the userinteraction to send the above interaction very streamlined and simple,such as by presenting a single option like “Send ‘Heading home’ in amessage to spouse?”, which requires only a single click from the user todo. In other implementations, generation and sending of the messagecould be performed completely autonomously.

Although FIG. 27 shows the present systems, devices, and methods asbeing utilized in a passenger car, one skilled in the art willappreciate that the present systems, devices, and methods are applicableto any appropriate vehicle, including but not limited to vans, trucks,buses, tractors, scooters, mopeds, motorcycles, airplanes, helicopters,boats, ships, other vessels, bicycles, and unicycles. FIG. 28illustrates an example scene 2800 as might be captured by a cameraincluded on a WHUD system worn by a user operating a motorcycle. Atleast one processor 301 included in the WHUD system can analyze visualdata captured by the camera to detect features represented in the visualdata which indicate whether a user is operating a vehicle. For example,the at least one processor 301 could identify equipment and featureswhich can be found on a motorcycle or similar vehicle, such ashandlebars 2801 and at least one user hand 2802 thereon, at least oneinformation indicator 2803, at least one side-view mirror 2804, controls2805, headlight 2806, windscreen 2807, or any other vehicle feature asappropriate. Information indicator 2803 could include for example aspeedometer, engine RPM gauge, engine temperature gauge, odometer, fuelgauge, status indication lights, warning lights, vehicle informationscreen, or any other indicator appropriate for a given application.Controls 2805 could include at least one user input interface forproviding user control of any features of the vehicle, such as buttons,knobs, switches, levers, touch-sensitive inputs, or any other userinterface as appropriate for a given application. In addition, the atleast one processor may also identify features external to the vehiclewhich can indicate whether a user of the WHUD system is operating avehicle, such as those described above with reference to FIG. 27 , inaddition to traffic light 2811, light post 2812, and road sign 2813. Oneskilled in the art will appreciate that these additional featuresexternal to the vehicle are independent of the vehicle, and thus can beidentified to determine whether a user is operating a vehicle in any ofthe implementations discussed herein.

Similarly to as discussed with reference to FIG. 27 above, theidentified features such as those listed above may not be completelyrepresented in the visual data. As an example, the captured visual datadoes not include a whole representation of the motorcycle which the useris operating. As another example, the captured visual data onlypartially represents light post 2812. As another example, the capturedvisual data only represents the user's hands 2802, and does notrepresent an entire body of the user. Nonetheless, the at least oneprocessor 301 can identify that the user is operating a motorcycle basedon identification of partially visible features.

After identification that the user is operating a motorcycle, the atleast one processor can generate at least one candidate message,optionally based on additional environmental context parameters of thedevice, using any of the techniques discussed herein, based on data fromany of the sensors discussed herein. The at least one processor 301 canalso determine at least one candidate recipient using any of thetechniques discussed herein, based on data from any of the sensorsherein. Similar messages and recipients could be determined as discussedwith reference to FIG. 27 above, but message content and recipientdetermination could also be based on what vehicle the user is operating.As an example, motorcycles for many people can be a leisure vehicle,such as for driving through the countryside. In such an example, aftercapturing visual data of a country road, the at least one processor 301may determine that the user's cloud storage is an appropriate recipientfor a message, and may generate message content like a simple timeand/or location stamp. Such a process may be performed completelyautonomously, so that the user may review their captured visual dataafter they are finished their ride, thus improving safety.

FIG. 29 is another exemplary scene 2900 as captured by a camera. Forexample, the camera could be carried by a WHUD worn by a user, such thatthe view of the camera closely approximates the view seen by the user.Scene 2900 can represent a user's workspace, such as in their office.The at least one processor 301 can determine that the user is currentlyin their workspace based on analysis of the captured visual data, suchas by identifying known features like monitor 2910, door 2911, doorknob2912 and clock 2913. The user may have previously registered thefeatures of their workspace by performing a camera capture of the area,or system 300 may have built a database of identifiable featuresautonomously over time since the user is regularly in their workspace.Preferably, autonomous construction of a database of identifiablefeatures would require permission of the user, in the interests ofmaintaining privacy of the user and privacy of the user's employer. Ascan be seen in FIG. 29 , monitor 2910 and door 2911 are only partiallyrepresented in the captured visual data, but even these partialrepresentations can be used by the at least one processor 301 toidentify the scene as a whole. Additionally, the at least one processormay also identify the location and activity of the user based on datafrom environmental context sensors beyond just a camera. For example,the at least one processor 301 may determine that the user is at theirplace of work based on location data from a location sensor. Further theat least one processor may be able to determine that the user isunfocused on work (e.g. bored), for example by an IMU of system 300detecting nodding motions of the user's head, or an eye-tracking deviceof system 300 detecting that the user's eyes are barely open or keepshutting for long periods of time as if falling asleep. The at least oneprocessor could also for example determine based on data from the IMUand/or the eye tracking device that the user's gaze is directed towardsclock 2913 as opposed to their monitor 2910.

Based on at least one of the determined location and/or activity of theuser, the at least one processor may generate candidate messages like“Bored at work” or “Can't wait to get home and see you!”. Further, theat least one processor 301 could also determine at least one candidaterecipient based on the determined location, activity, message content,or any other environmental context data. In the above example, the atleast one processor 301 could determine that the content of the messageis inappropriate for sending to work colleagues, and thus insteaddetermine close friends or family as candidate recipients for themessage. As another example, if the user selects a message like “Can'twait to get home and see you!”, the at least one processor 301 candetermine that the most appropriate candidate recipients for the messageare people who live in the same home as the user.

Further, although in the above examples candidate messages are generatedthen candidate recipients are determined, this order of operations couldbe changed, such that candidate recipients are determined, thencandidate messages are generated. In some implementations, generation ofcandidate message and determination of candidate recipients can besubstantially simultaneous.

As is evident from the above discussion regarding FIGS. 4, 5A, 5B, 6, 7,8A-8C, 9A, 9B, 10A-10C, 11, 12A-12C, 13, 14A-14D, 15A-15D, 16, 17 ,18A-18D, 19A-19D, 20A-20D, 21A-21E, 22A-22D, 23A-23D, 24A-24D, 25A-25C,26A-26C, 27, 28, and 29, it is within the scope of the presentapplication to reorder, combine, or remove any acts related togeneration of candidate messages and candidate recipients, as well asany acts of receiving user input and selection as is appropriate for aspecific application.

In the above discussed implementations of FIGS. 5A-5B, 8A-8C, 9A-9B,10A-10C, 12A-12C, 14A-14D, 15A-15D, 18A-18D, 19A-19D, 20A-20D, 21A-21E,22A-22D, 23A-23D, 24A-24D, 25A-25C, and 26A-26C, the letter-labelledsub-figures represent a sequence of user interfaces which can bepresented or displayed to the user, starting from A onwards, however theorder of interfaces could be changed as appropriate for a givenapplication. FIG. 5A represents a first user interface presented to auser, and FIG. 5B represents a subsequent second user interfacepresented to the user subsequent to presentation the first userinterface. FIG. 8A represents a first user interface presented to auser, FIG. 8B represents a subsequent second user interface presented tothe user after presentation of the first user interface, and FIG. 8Crepresents a subsequent third user interface presented to the user afterpresentation of the second user interface. FIG. 9A represents a firstuser interface presented to a user, and FIG. 9B represents a subsequentsecond user interface presented to the user after presentation of thefirst user interface. FIG. 12A represents a first user interfacepresented to a user, FIG. 12B represents a second subsequent userinterface presented to the user after presentation of the first userinterface, and FIG. 12C represents a subsequent third user interfacepresented to the user after presentation of the second user interface.FIG. 14A represents a first user interface presented to a user, FIG. 14Brepresents a subsequent second user interface presented to the userafter presentation of the first user interface, FIG. 14C represents asubsequent third user interface presented to the user after presentationof the second user interface, and FIG. 14D represents a subsequentfourth user interface presented to the user after presentation of thethird user interface. FIG. 15A represents a first user interfacepresented to a user, FIG. 15B represents a subsequent second userinterface presented to the user after presentation of the first userinterface, FIG. 15C represents a subsequent third user interfacepresented to the user after presentation of the second user interface,and FIG. 15D represents a subsequent fourth user interface presented tothe user after presentation of the third user interface. FIG. 18Arepresents a first user interface presented to a user, FIG. 18Brepresents a subsequent second user interface presented to the userafter presentation of the first user interface, FIG. 18C represents asubsequent third user interface presented to the user after presentationof the second user interface, and FIG. 18D represents a subsequentfourth user interface presented to the user after presentation of thethird user interface. FIG. 19A represents a first user interfacepresented to a user, FIG. 19B represents a subsequent second userinterface presented to the user after presentation of the first userinterface, FIG. 19C represents a subsequent third user interfacepresented to the user after presentation of the second user interface,and FIG. 19D represents a subsequent fourth user interface presented tothe user after presentation of the third user interface. FIG. 20Arepresents a first user interface presented to a user, FIG. 20Brepresents a subsequent second user interface presented to the userafter presentation of the first user interface, FIG. 20C represents asubsequent third user interface presented to the user after presentationof the second user interface, and FIG. 20D represents a subsequentfourth user interface presented to the user after presentation of thethird user interface. FIG. 21A represents a first user interfacepresented to a user, FIG. 21B represents a subsequent second userinterface presented to the user after presentation of the first userinterface, FIG. 21C represents a subsequent third user interfacepresented to the user after presentation of the second user interface,FIG. 21D represents a subsequent fourth user interface presented to theuser after presentation of the third user interface, and FIG. 21Erepresents a subsequent fifth user interface presented to the user afterpresentation of the fourth user interface. FIG. 22A represents a firstuser interface presented to a user, FIG. 22B represents a subsequentsecond user interface presented to the user after presentation of thefirst user interface, FIG. 22C represents a subsequent third userinterface presented to the user after presentation of the second userinterface, and FIG. 22D represents a subsequent fourth user interfacepresented to the user after presentation of the third user interface.FIG. 23A represents a first user interface presented to a user, FIG. 23Brepresents a subsequent second user interface presented to the userafter presentation of the first user interface, FIG. 23C represents asubsequent third user interface presented to the user after presentationof the second user interface, and FIG. 23D represents a subsequentfourth user interface presented to the user after presentation of thethird user interface. FIG. 24A represents a first user interfacepresented to a user, FIG. 24B represents a subsequent second userinterface presented to the user after presentation of the first userinterface, FIG. 24C represents a subsequent third user interfacepresented to the user after presentation of the second user interface,and FIG. 24D represents a subsequent fourth user interface presented tothe user after presentation of the third user interface. FIG. 25Arepresents a first user interface presented to a user, FIG. 25Brepresents a subsequent second user interface presented to the userafter presentation of the first user interface, and FIG. 25C representsa subsequent third user interface presented to the user afterpresentation of the second user interface. FIG. 26A represents a firstuser interface presented to a user, FIG. 26B represents a subsequentsecond user interface presented to the user after presentation of thefirst user interface, and FIG. 26C represents a subsequent third userinterface presented to the user after presentation of the second userinterface.

Further, FIGS. 5A-5B, 8A-8C, 9A-9B, 10A-10C, 12A-12C, 14A-14D, 18A-18D,19A-19D, 20A-20D, 21A-21E, 22A-22D, 23A-23D, 24A-24D, and 26A-26C eachshow visual interfaces which could be presented to the user; however,any of these interfaces could be replaced by alternative presentationschemes, as appropriate for a given application. As an example, insteadof presenting the interfaces in the above figures visually using adisplay, system 300 could instead present each interface as an audiointerface, in which options, candidate messages, candidate recipients,and any other important information is read out to the user as audio.

One skilled in the art will appreciate that each of the describedhardware, sensors, and techniques used for identifying an environmentalcontext parameter, and generating messages based on the environmentalcontext, are applicable to all described implementations of the subjectsystems, devices, and methods. For example, one skilled in the art willrecognize that the hardware, sensors, and techniques used for generatingcandidate messages as described with relation to FIGS. 4, 5A-5B, 6, 7,8A-8C, 9A-9B, and 10A-10C are fully applicable to the generation of afirst portion of a message and at least one second portion to completethe message as described with regards to FIGS. 11, 12A-12C, 13, 14A-14D,and 151-15D, and are fully applicable to implementations where visualdata is captured and used to generate a message as described withregards to FIGS. 16, 17, 18A-18D, 19A-19D, 20A-20D, 21A-21E, 22A-22D,23A-23D, 24A-24D, 25A-25C, 26A-26C, 27 , 28, and 29. Further, oneskilled in the art will appreciate that each of the described hardware,sensors, and techniques for determining a recipient to send a message toare fully applicable to all implementations of the subject systems,devices, and methods. For example, the hardware, sensors, and techniquesfor determining a recipient as described with relation to the method ofFIG. 4 are fully applicable to the methods of FIGS. 7, 11, 13, 16, and17 . Similarly, the hardware, sensors, and techniques for determining arecipient as described with relation to the method of FIG. 7 are fullyapplicable to the methods of FIGS. 4, 11, 13, 16, and 17 . Similarly,the hardware, sensors, and techniques for determining a recipient asdescribed with relation to the method of FIG. 11 are fully applicable tothe methods of FIGS. 4, 7, 13, 16, and 17 . Similarly, the hardware,sensors, and techniques for determining a recipient as described withrelation to the method of FIG. 13 are fully applicable to the methods ofFIGS. 4, 7, 11, 16, and 17 . Similarly, the hardware, sensors, andtechniques for determining a recipient as described with relation to themethods of FIGS. 16 and 17 are fully applicable to the methods of FIGS.4, 7, 11, and 13 .

One skilled in the art will appreciate that all of the generatedcandidate message or all of the determined candidate recipients, whendisplayed, do not need to be displayed on a single display, but ratherthe user may be able to scroll or click or otherwise iterate or navigatethrough multiple user interfaces presenting different candidate messagesor different determined candidate recipients. Further still, in any ofthe above implementations, the user interfaces presented on outputdevice 304 may also include user selectable icons or options to generatedifferent candidate messages or determine different candidate recipientsfor the message. In this way, if a user is unsatisfied with thecurrently presented candidate messages or the currently presentedcandidate recipients, the user may provide input to generate differentcandidate messages or determine different candidate recipients.Additionally, in implementations where a user selects an icon thatcorresponds to an action such as “Compose message” to enter the messagecomposition process or mode, the “Compose message” option as displayedon the main user interface presented via a display of system 300 mayinclude a preview of one of the candidate messages or candidaterecipients as generated by the at least one processor 301.

Some significant advantages to using messaging schemes like thoseillustrated in FIGS. 4, 5A-5B, 6, 7, 8A-8C, 9A-9B, 10A-10C, 11, 12A-12C,13, 14A-14D, 15A-15D, 16, 17 , 18A-18D, 19A-19D, 20A-20D, 21A-21E,22A-22D, 23A-23D, 24A-24D, 25A-25C, 26A-26C, 27, 28, and 29 is speed,ease, and privacy for the user. Specifically, all of the above schemesperform most of the text creation for the user, with the user only beingrequired to select at least one or maybe a small number of options. Thisenables the user to quickly create and send messages, with minimal timeand thought investment, and without requiring the user to use aconvoluted or inefficient text input scheme as many portable electronicdevices require. Further, these schemes can be implemented without theneed for voice commands, thus maintaining the privacy of text-basedcommunication. One skilled in the art would appreciate, however, thatthe present systems, methods, and devices could employ voice commands ifdesired, at the expense of privacy. Further, the messaging schemes aboveadvantageously enable the user to send messages largely or completelyhands free. For example, the user may only need to click a button on aring a few times (or even just one time) to send a message, instead ofhaving to pull out a smartphone, open a messaging application, type amessage, choose a recipient, and send the message. In someimplementations, input can be completely hands-free, such as byreceiving a voice command like “send message 1”, where message 1corresponds to a message shown to the user on a display. Such aninterface can maintain some privacy of the user, since they are notrequired to say the content of the message, while also enablingcompletely hands-free input.

In any of the above discussed implementations, an optional act may beadded where a user inputs a theme of a message before the at least oneprocessor 301 generates a message. For example, an output device ofsystem 300 may present to the user a number of different options formessage theme generated by the at least one processor 301, and the usermay select one of the options before the at least one processor 301generates candidate messages. As an example, in the above discussed“explore” mode, the output device could present to the user a list ofdifferent venue types, such as “restaurants”, “art galleries”,“theatres”, “nature parks”, “museums”, “concerts”, and “festivals”. Ifthe user selects “restaurants”, the at least one processor 301 maysearch for restaurants within a one-kilometer radius for the user toselect. If the user selects “art galleries”, the at least one processor301 may search for art galleries within a 20-kilometer radius of theuser. As another example, in the implementations of FIGS. 11 and 12 ,the output device could present a number of message themes like “Work”,“Home”, “Kids”, or any other appropriate theme. In response to a userselection of a theme, the at least one processor 301 can generatecandidate messages which are highly relevant to the selected theme.Further, the options for message themes may be a text strings statingthemes, or may be images representing themes such as emojis, emoticons,photographs, computer generated renderings, or any other type of imageas appropriate for a specific application.

In any of the above discussed implementations, an option can beavailable to the user to view more candidate messages and/or morecandidate recipients. For example, all of the generated candidatemessages may not fit on a single page of the display, and the user maybe provide the option to scroll through a list or set of all of thepotential candidate messages. As an example, if the at least oneprocessor 301 generates ten candidate messages, but the display can onlyshow three candidate messages at a time in a vertical list, the usercould provide a “down” directional input (e.g., select down arrow icon,swipe up) to move through candidate messages in the list. Once the userreaches the bottom of the displayed candidate messages (in this case,the third message), the user may continue to provide a “down” directioninput (e.g., select down arrow icon again, continue to swipe up) toswitch to another page of three different candidate messages. As anotherexample, if the display can only display three candidate messages in avertical list, the user could provide a “right” or “left” input toswitch to other pages showing different candidate messages.

Further, an option may be provided to generate new candidate messages,for example, if the user is unsatisfied with all of the alreadygenerated candidate messages. For example, the display may display a“Generate more” user selectable icon, which the user can select and/orclick on to cause generation or a presentation of additional candidatemessages. Alternatively, the system may be receptive to a voice commandfrom the user such as “Generate more”. In response to a user instructionto generate more candidate messages, the at least one processor 301 maylog the previously generated messages, and repeat the generationprocesses, excluding the options that were already generated. Further,the at least one processor 301 may also analyze trends among therejected messages to improve the relevance of the candidate messagesgenerated in the second round. As an example, if the at least oneprocessor 301 generates a first round of candidate messages which areall related to the user's professional life, and the at least oneprocessor 301 receives an input indicating that the user wants differentcandidate messages, the at least one processor 301 may change the themefor generation and generate candidate messages pertaining to the user'spersonal life.

In the present systems, devices and methods, most of the acts can beperformed autonomously, without the need for user input, unlessexplicitly specified otherwise. For example, in method 700, the at leastone processor can autonomously perform acts 701, 702, and 705; theoutput device can autonomously perform acts 703 and 706; and thewireless communications module can autonomously perform act 708.Regarding acts 704 and 707, input from the user is required, butreception and handling of this input is performed autonomously by the atleast one processor. This discussion also applies to the implementationsshown in FIGS. 4, 11, 13, 16, and 17 , such that the hardware whichperforms each act can perform the respective act autonomously.

Many of the figures and descriptions herein pair a candidate recipientwith the relationship between the recipient and the user. For example,recipients “Husband Bert”, “Boss Jean”, “Coworker Bill”, “SecretaryFerdinand”, “Dad”, “Uncle George”, “Aunt Tina”, and “Daughter Mildred”all include a specification of the relationship between the user and therecipient, to make the description more clear. However, such arelationship does not necessarily need to be included in thepresentation of the recipient as a candidate recipient. For example,“Boss Jean” could instead be presented as “Jean Willow”, which is BossJean's first and last name. Data regarding the relationship between acandidate recipient and the user can be stored as meta-data for therecipient. Such metadata could be input by the user, or could beinferred by the at least one processor 301 over time. Further, in someimplementations the relationship between a candidate recipient and theuser does not need to be explicitly specified.

Throughout this specification and the appended claims the term“communicative” as in “communicative coupling,” and in variants such as“communicatively coupled,” is generally used to refer to any engineeredarrangement for transferring and/or exchanging information. Exemplarycommunicative pathways include, but are not limited to, electricallyconductive pathways (e.g., electrically conductive wires, electricallyconductive traces), magnetic pathways (e.g., magnetic media), and/oroptical pathways (e.g., optical fiber), and exemplary communicativecouplings include, but are not limited to, electrical couplings,magnetic couplings, and/or optical couplings.

Throughout this specification and the appended claims, infinitive verbforms are often used. Examples include, without limitation: “to detect,”“to provide,” “to transmit,” “to communicate,” “to process,” “to route,”and the like. Unless the specific context requires otherwise, suchinfinitive verb forms are used in an open, inclusive sense, that is as“to, at least, detect,” to, at least, provide,” “to, at least,transmit,” and so on.

The above description of illustrated embodiments, including what isdescribed in the Abstract, is not intended to be exhaustive or to limitthe embodiments to the precise forms disclosed. Although specificembodiments of and examples are described herein for illustrativepurposes, various equivalent modifications can be made without departingfrom the spirit and scope of the disclosure, as will be recognized bythose skilled in the relevant art. The teachings provided herein of thevarious embodiments can be applied to other portable and/or wearableelectronic devices, not necessarily the exemplary wearable electronicdevices generally described above.

For instance, the foregoing detailed description has set forth variousembodiments of the systems and/or processes via the use of blockdiagrams, schematics, and examples. Insofar as such block diagrams,schematics, and examples contain one or more functions and/oroperations, it will be understood by those skilled in the art that eachfunction and/or operation within such block diagrams, flowcharts, orexamples can be implemented, individually and/or collectively, by a widerange of hardware, software, firmware, or virtually any combinationthereof. In one embodiment, the present subject matter may beimplemented via Application Specific Integrated Circuits (ASICs).However, those skilled in the art will recognize that the embodimentsdisclosed herein, in whole or in part, can be equivalently implementedin standard integrated circuits, as one or more computer programsexecuted by one or more computers (e.g., as one or more programs runningon one or more computer systems), as one or more programs executed by onone or more controllers (e.g., microcontrollers) as one or more programsexecuted by one or more processors (e.g., microprocessors, centralprocessing units, graphical processing units), as firmware, or asvirtually any combination thereof, and that designing the circuitryand/or writing the code for the software and or firmware would be wellwithin the skill of one of ordinary skill in the art in light of theteachings of this disclosure.

When logic is implemented as software and stored in memory, logic orinformation can be stored on any processor-readable storage medium foruse by or in connection with any processor-related system or method. Inthe context of this disclosure, a memory is a processor-readable storagemedium that is an electronic, magnetic, optical, or other physicaldevice or means that contains or stores a computer and/or processorprogram. Logic and/or the information can be embodied in anyprocessor-readable storage medium for use by or in connection with aninstruction execution system, apparatus, or device, such as acomputer-based system, processor-containing system, or other system thatcan fetch the instructions from the instruction execution system,apparatus, or device and execute the instructions associated with logicand/or information.

In the context of this specification, a “non-transitoryprocessor-readable storage medium” can be any element that can store theprogram associated with logic and/or information for use by or inconnection with the instruction execution system, apparatus, and/ordevice. The processor-readable storage medium can be, for example, butis not limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus or device. More specificexamples (a non-exhaustive list) of the processor-readable storagemedium would include the following: a portable computer diskette(magnetic, compact flash card, secure digital, or the like), a randomaccess memory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM, EEPROM, or Flash memory), a portable compactdisc read-only memory (CDROM), digital tape, and other non-transitorymedia.

The various embodiments described above can be combined to providefurther embodiments. To the extent that they are not inconsistent withthe specific teachings and definitions herein, all of the following areincorporated herein by reference in their entirety: U.S. ProvisionalPatent Application Ser. No. 62/692,484, U.S. Non-Provisional patentapplication Ser. No. 15/145,576, U.S. Non-Provisional patent applicationSer. No. 15/167,458, U.S. Non-Provisional patent application Ser. No.15/046,254, U.S. Provisional Patent Application Ser. No. 62/607,816,U.S. Provisional Patent Application Ser. No. 62/607,819, U.S.Provisional Patent Application Ser. No. 62/671,248, U.S. Non-Provisionalpatent application Ser. No. 14/520,081, U.S. Non-Provisional patentapplication Ser. No. 14/567,826, U.S. Non-Provisional patent applicationSer. No. 14/737,081, U.S. Non-Provisional patent application Ser. No.15/167,458, U.S. Non-Provisional patent application Ser. No. 15/827,667,U.S. Provisional Patent Application Ser. No. 62/658,436, and U.S.Provisional Patent Application Ser. No. 62/750,060. Aspects of theembodiments can be modified, if necessary, to employ systems, circuitsand concepts of the various patents, applications and publications toprovide yet further embodiments.

These and other changes can be made to the embodiments in light of theabove-detailed description. In general, in the following claims, theterms used should not be construed to limit the claims to the specificembodiments disclosed in the specification and the claims, but should beconstrued to include all possible embodiments along with the full scopeof equivalents to which such claims are entitled. Accordingly, theclaims are not limited by the disclosure.

1. A first system which initiates a communication between the firstsystem and a second system, the first system comprising: at least oneprocessor; an output device communicatively coupled to the at least oneprocessor; a wireless communications module communicatively coupled tothe at least one processor; and a non-transitory processor-readablestorage medium communicatively coupled to the at least one processor;wherein the non-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to: identify, by the at least oneprocessor, an environmental context parameter; generate, by the at leastone processor, at least one candidate message based on the environmentalcontext parameter; cause a presentation, by the output device, of the atleast one candidate message; receive, by the at least one processor, afirst input from a user of the first system, the first input indicativeof a selected message from the at least one candidate message; and causea transmission, by the wireless communications module, of the selectedmessage.
 2. The first system of claim 1, further comprising: a wearableheads-up display (“WHUD”) which comprises the at least one processor, adisplay communicatively coupled to the at least one processor, a firstwireless communications module communicatively coupled to the at leastone processor, and the non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; and a peripheraldevice comprising a second wireless communications module; wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message, cause the first system to:transmit, by the first wireless communications module of the WHUD, theselected message; receive, by the second wireless communications moduleof the peripheral device, the selected message; and transmit, by thesecond wireless communications module of the peripheral device, theselected message.
 3. The first system of claim 1 wherein theenvironmental context parameter represents an environmental context ofthe first system or an environmental context of a user of the firstsystem.
 4. The first system of claim 1, further comprising a clockcommunicatively coupled to the at least one processor, wherein: theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to identify, by the at least oneprocessor, an environmental context parameter cause the at least oneprocessor to identify at least one of a time of day or a date based onan output of the clock; and the processor-executable instructions that,when executed by the at least one processor, cause the first system togenerate, by the at least one processor, at least one candidate messagecause the at least one processor to generate at least one greetingappropriate for the at least one of the time of day or the date.
 5. Thefirst system of claim 1, further comprising a clock communicativelycoupled to the at least one processor, wherein: the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify a time of day and a date based on an output of the clock andidentify at least one relevant calendar entry in a calendar based on thetime of day and the date; and the processor-executable instructionsthat, when executed by the at least one processor, cause the firstsystem to generate, by the at least one processor, at least onecandidate message cause the at least one processor to generate at leastone candidate message based on the at least one relevant calendar entry.6. The first system of claim 1 wherein: the processor-executableinstructions that, when executed by the at least one processor, causethe first system to identify, by the at least one processor, anenvironmental context parameter cause the at least one processor toidentify an activity performed by the user of the first system based oninput sensor data received from a sensing device; and theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to generate, by the at least oneprocessor, at least one candidate message cause the at least oneprocessor to generate at least one candidate message based on theactivity performed by the user of the first system.
 7. The first systemof claim 6 wherein the at least one input sensor includes at least onesensor selected from a group consisting of: an electromyography sensor,a mechanomyography sensor, an accelerometer, a gyroscope, anelectrocardiography sensor, a photoplethysmography sensor, a bloodpressure sensor, a global positioning system sensor, a compass, and aneye tracking sensor.
 8. The first system of claim 1, further comprisinga location sensor communicatively coupled to the at least one processor,wherein: the processor-executable instructions that, when executed bythe at least one processor, cause the first system to identify, by theat least one processor, an environmental context parameter cause the atleast one processor to identify a location of the first system based onan output of the location sensor; and the processor-executableinstructions that, when executed by the at least one processor, causethe first system to generate, by the at least one processor, at leastone candidate message cause the at least one processor to generate atleast one candidate message based on the location of the first system.9. The first system of claim 1, further comprising at least one physicalenvironment sensor communicatively coupled to the at least oneprocessor, wherein: the processor-executable instructions that, whenexecuted by the at least one processor, cause the first system toidentify, by the at least one processor, an environmental contextparameter cause the at least one processor to identify at least aphysical environment of the first system based on an output of thephysical environment sensor; and the processor-executable instructionsthat, when executed by the at least one processor, cause the firstsystem to generate, by the at least one processor, at least onecandidate message cause the at least one processor to generate at leastone candidate message based on the physical environment of the firstsystem.
 10. The first system of claim 9 wherein the at least onephysical environment sensor includes at least one sensor selected from agroup consisting of: a camera and a microphone.
 11. The first system ofclaim 1 wherein the non-transitory processor-readable storage mediumstores further processor-executable instructions that, when executed bythe at least one processor, cause the first system to: determine, by theat least one processor, at least one candidate recipient for theselected message based on at least one of a content of the selectedmessage and the environmental context parameter; present, by the outputdevice, the at least one candidate recipient for the selected message;and receive, by the at least one processor, a second input from the userof the first system, the second input indicative of a selected recipientfrom the at least one candidate recipient, wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message cause the wirelesscommunications module to transmit the selected message to the selectedrecipient.
 12. The first system of claim 11 wherein the non-transitoryprocessor-readable storage medium stores further processor-executableinstructions that, when executed by the at least one processor, causethe first system to modify, by the at least one processor, the selectedmessage based on the selected recipient.
 13. The first system of claim11 wherein the non-transitory processor-readable storage medium storesfurther processor-executable instructions that, when executed by the atleast one processor, cause the first system to determine, by the atleast one processor, a communication vernacular based on a communicationhistory between the user of the first system and the selected recipient,wherein the processor-executable instructions that, when executed by theat least one processor, cause the first system to generate, by the atleast one processor, at least one candidate message cause the at leastone processor to generate at least one candidate message based on thecommunication vernacular.
 14. The first system of claim 11 wherein thenon-transitory processor-readable storage medium stores furtherprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to determine, by the at least oneprocessor, a relationship between the user of the first system and theselected recipient, wherein the processor-executable instructions that,when executed by the at least one processor, cause the first system togenerate, by the at least one processor, at least one candidate messagecause the at least one processor to generate at least one candidatemessage based on the determined relationship.
 15. The first system ofclaim 1 wherein the non-transitory processor-readable storage mediumstores further processor-executable instructions that, when executed bythe at least one processor, cause the first system to: prior togeneration, by the at least one processor, of at least one candidatemessage: determine, by the at least one processor, at least onecandidate recipient based on the environmental context parameter;present, by the output device, the at least one candidate recipient; andreceive, by the at least one processor, a second input from the user ofthe first system, the second input indicative of a selected recipientfrom the at least one candidate recipient, wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message cause the wirelesscommunications module to transmit the selected message to the selectedrecipient.
 16. The first system of claim 1, wherein: theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to generate, by the at least oneprocessor, at least one candidate message based on the environmentalcontext parameter cause the at least one processor to generate at leasttwo candidate messages based on the environmental context parameter; andthe processor-executable instructions that, when executed by the atleast one processor, cause the first system to receive, by the at leastone processor, a first input from a user of the first system, cause theat least one processor to receive a first input indicative of a selectedmessage from the at least two candidate messages.
 17. The first systemof claim 1 wherein: the non-transitory processor-readable storage mediumstores further processor-executable instructions that, when executed bythe at least one processor, cause the first system to generate, by theat least one processor, a first portion of a message; theprocessor-executable instructions which cause the first system togenerate, by the at least one processor, at least one candidate messagecause the at least one processor to generate at least two candidatesecond portions to complete the message based on at least the firstportion of the message; the processor-executable instructions whichcause the first system to receive, by the at least one processor, afirst input from a user of the first system cause the at least oneprocessor to receive a first input from a user of the first systemindicative of a selected second portion from the at least two candidatesecond portions; and the non-transitory processor-readable storagemedium stores further processor-executable instructions that, whenexecuted by the at least one processor, cause the first system togenerate, by the at least one processor, a complete message by combiningthe first portion and the selected second portion.
 18. A first systemwhich initiates a communication between the first system and a secondsystem, the first system comprising: at least one processor; an outputdevice communicatively coupled to the at least one processor; a wirelesscommunications module communicatively coupled to the at least oneprocessor; and a non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; wherein thenon-transitory processor-readable storage medium storesprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to: generate, by the at least oneprocessor, a first portion of a message; present, by the output device,the first portion of the message; generate, by the at least oneprocessor, at least two candidate second portions to complete themessage based on at least the first portion of the message; present, bythe output device, the at least two candidate second portions; receive,by the at least one processor, a first input from a user of the firstsystem, the first input indicative of a selection of one the at leasttwo candidate second portions as a selected second portion; generate, bythe at least one processor, a complete message by combining the firstportion and the selected second portion; and transmit, by the wirelesscommunications module, the complete message from the first system. 19.The first system of claim 18, further comprising: a wearable heads-updisplay (“WHUD”) which comprises the at least one processor, a displaycommunicatively coupled to the at least one processor, a first wirelesscommunications module communicatively coupled to the at least oneprocessor, and the non-transitory processor-readable storage mediumcommunicatively coupled to the at least one processor; and a peripheraldevice comprising a second wireless communications module; wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the selected message, cause the first system to:transmit, by the first wireless communications module of the WHUD, thecomplete message; receive, by the second wireless communications moduleof the peripheral device, the complete message; and transmit, by thesecond wireless communications module of the peripheral device, thecomplete message.
 20. The first system of claim 18 wherein at least oneof the at least two candidate second portions is a text string, and theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to present, by the output device,the at least two candidate second portions cause the output device topresent the text string.
 21. The first system of claim 18 wherein atleast one of the at least two candidate second portions is a non-textimage, and the processor-executable instructions that, when executed bythe at least one processor, cause the first system to present, by theoutput device, the at least two candidate second portions cause theoutput device to display the image.
 22. The first system of claim 18wherein: the non-transitory processor-readable storage medium storesfurther processor-executable instructions that, when executed by the atleast one processor, cause the at least one processor to identify anenvironmental context parameter, the environmental context parameterrepresenting an environmental context including at least one contextselected from a group consisting of: a time, a date, a calendar entry ina calendar, a location, a physical environment, and an activityperformed by the user; the processor-executable instructions that, whenexecuted by the at least one processor, cause the first system generate,by the at least one processor, a first portion of a message cause the atleast one processor to generate the first portion of the message basedon the environmental context parameter; and the processor-executableinstructions that, when executed by the at least one processor, causethe first system to generate, by the at least one processor, at leasttwo candidate second portions to complete the message based on at leastthe first portion of the message cause the at least one processor togenerate the at least two candidate second portions based on theenvironmental context parameter.
 23. The first system of claim 18wherein the non-transitory processor-readable storage medium storesfurther processor-executable instructions that, when executed by the atleast one processor, cause the at least one processor to autonomouslydetermine at least one recipient of the complete message based on atleast one of a content of the complete message and the environmentalcontext parameter.
 24. The first system of claim 18 wherein thenon-transitory processor-readable storage medium stores furtherprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to: determine, by the at least oneprocessor, at least one candidate recipient for the complete messagebased on at least one of a content of the complete message and theenvironmental context parameter; present, by the output device, the atleast one candidate recipient for the complete message; and receive, bythe at least one processor, a second input from the user of the firstsystem, the second input indicative of a selection of one of the atleast one candidate recipient as a selected recipient, wherein theprocessor-executable instructions that, when executed by the at leastone processor, cause the first system to transmit, by the wirelesscommunications module, the complete message cause the wirelesscommunications module to transmit the complete message to the selectedrecipient.